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Executive  Summary 


The  Canadian  Forces  (CF)  trains  in  a  wide  variety  of  skills.  For  reasons  of  efficiency  and  economy,  it 
is  important  to  conduct  no  more  refresher  training  than  is  necessary  to  keep  performance  at  the  desired 
skill  level.  The  purpose  of  this  contract  is  to  obtain  a  survey  of  the  scientific  and  technical  literature 
on  models  of  skill  fading  and  tools  for  determining  when  refresher  training  is  required.  In  particular, 
the  literature  review  investigated  models  for  predicting  skill  retention  relevant  to  the  military  domain. 

Skill  retention  can  be  defined  as  the  maintenance  or  sustainment  of  skills  as  learned  behaviors  and 
procedures  over  long  periods  of  time  without  practice  (Schendel  &  Hagman,  1991).  Degradation  in 
performance  can  be  observed  because  the  perceptual,  motor,  and  cognitive  processes  that  underlie 
skilled  performance  decay  or  break-down  or  because  the  individual  loses  the  ability  to  access  or 
perform  those  processes. 

Skill  retention  can  be  described  by  a  power  function  relating  level  of  performance  to  the  retention 
interval  between  training  and  test.  This  function  is  negatively  accelerated,  meaning  that  performance 
declines  most  rapidly  soon  after  learning  then  at  an  increasingly  slow  rate  over  time.  The  two 
parameters  by  which  skill  loss  can  be  characterized  are  the  slope  of  the  curve,  describing  the  rate  of 
skill  loss,  and  the  asymptote,  which  describes  the  level  at  which  performance  levels  out.  The  power 
function  offers  the  most  accurate  description  of  retention  for  a  range  of  tasks  and  materials. 

This  literature  review  examines  eight  factors  demonstrated  to  affect  skill  retention: 

•  Retention  Interval. 

•  Opportunity  to  practice. 

•  Degree  of  learning. 

•  Method  of  training. 

•  Similarity  of  training  and  performance  environments. 

•  Type  of  task 

•  Method  of  testing. 

•  Individual  differences. 

The  literature  review  then  examines  several  classes  of  models  for  predicting  skill  retention.  Subjective 
approaches  involve  some  form  of  self-assessment  of  retention  and/or  need  for  refresher  training  on  the 
part  of  trained  individuals.  These  techniques  can  yield  accurate  estimates  of  retention  but  are  often 
mistrusted.  Qualitative  approaches  indicate  categorically  how  skills  fade  over  time  in  relation  to 
certain  key  factors.  These  models,  as  they  stand  to  date,  do  not  solve  the  practical  problems  of 
predicting  when  proficiency  of  skills  will  decline  below  a  criterion  or  when  refresher  training  will  be 
needed  but  they  do  offer  approaches  that  may  be  developed  into  quantitative  models  in  the  future. 
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The  most  fully  developed  quantitative  model  of  skill  retention  is  the  U.S.  Army  Research  Institute’s 
Users'  Decision  Aid  (UDA)  model.  This  model  was  developed  specifically  to  provide  quantitative 
predictions  of  skill  retention  for  military  tasks.  The  UDA  model  was  based  on  empirical  studies 
documenting  factors  that  affect  skill  retention.  ARI  researchers  selected  the  following  factors  related 
to  the  task  because  these  can  be  conveniently  and  economically  measured: 

•  Number  of  steps 

•  Whether  steps  must  be  performed  in  a  set  sequence 

•  Whether  the  task  contains  feedback  that  indicates  the  correct  performance  of  steps 

•  Number  of  facts  or  information  chunks  that  must  be  recalled 

•  Execution  demands 

•  Whether  the  skill  is  cognitive  or  perceptual/motor 

•  Whether  there  are  job  and/or  memory  aids  for  the  task 

•  The  time  limit  of  the  task  (if  any) 

The  UDA  was  developed  by  an  iterative  process  of  determining  the  empirical  relationship  between  the 
set  of  factors  and  observed  retention  of  certain  military  skills  and  determining  the  best  fitting  function 
describing  that  relationship  (Rose  et  al.,  1985b): 

1.  Identify  task  dimensions  most  likely  to  be  related  to  retention. 

2.  Convert  task  dimensions  into  rating  scales,  develop  anchor  points,  and  analytically 
assign  weights  to  each  point  on  the  scales. 

3.  Assess  each  scale’ s  reliability  and  validity  by  having  several  judges  rate  tasks  on 
each  scale. 

4.  Examine  inter-rate  agreement  and  correlation  between  task  ratings  and  actual 
retention  data. 

5.  Iterate  these  steps  to  develop  a  set  of  valid  and  reliable  scales. 

The  UDA  model  was  developed  to  form  the  basis  of  a  decision  aid  tool  to  be  used  in  predicting  skill 
retention  and  the  need  for  refresher  training  in  military  tasks.  The  UDA  contains  ten  questions  that 
raters  answer  based  on  the  task  summary  and  their  knowledge  of  the  task.  Raters  select  the 
appropriate  answer  and  note  the  scale  value  associated  with  the  selected  answer.  When  all  ten 
questions  have  been  answered,  the  raters  compute  the  total  of  the  scale  values,  which  constitutes  the 
task’s  retention  score.  Multiple  raters  review  the  ratings  given  to  the  task  and  resolve  differences  to 
produce  a  final,  agreed-upon  task  retention  score.  Scores  are  interpreted  in  two  Performance 
Prediction  Tables  that  are  used  to  convert  the  total  retention  score  into  a  prediction  of  performance  for 
the  rated  task. 

The  UDA  is  one  of  few  approaches  for  which  empirical  research  has  been  done  to  assess  its 
applicability  and  practicality.  The  UDA  has  been  demonstrated  to  be  relatively  easy  to  administer, 
although  some  training  is  needed  to  perform  the  ratings  correctly  and  reliably.  The  UDA  itself  is  low- 
cost  and  requires  no  special  equipment.  The  UDA,  however,  has  received  empirical  validation  from 
just  one  study  that  reports  a  comparison  between  UDA  predictions  and  actual  retention  data. 

Other  quantitative  approaches  involve  determining  empirical  retention  curves  for  tasks  or  using 
training,  individual,  and  other  task  factors  to  predict  retention.  These  approaches  may  offer  useful 
concepts  but  require  extensive  effort  to  develop. 

The  literature  review  identifies  a  set  of  lessons  learned  and  discusses  five  specific  recommendations: 

1 .  Validate  the  UDA  for  the  CF. 
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2.  Conduct  research  to  determine  what  factors  can  be  incorporated  to  increase  the  accuracy 
of  predictions. 

3.  Develop  self-assessment  tools. 

4.  Determine  the  applicability  of  the  Relapse  Prevention  Model  to  supervisory, 
administrative,  and  other  related  tasks. 

5.  Conduct  studies  to  derive  empirical  retention  curves  for  critical  tasks. 
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1.  Introduction 


1.1  Background 

The  Canadian  Forces  (CF)  trains  in  a  wide  variety  of  skills,  ranging  from  perceptual-motor  skills,  such 
as  rifle  marksmanship,  to  predominantly  cognitive  skills,  such  as  command  and  control.  Initial 
training  raises  performance  to  a  criterion  level,  then  refresher  training  is  conducted  periodically  to 
maintain  performance  at  or  above  criterion.  For  reasons  of  efficiency  and  economy,  it  is  important  to 
conduct  no  more  refresher  training  than  is  necessary  to  keep  performance  at  the  desired  skill  level. 

The  purpose  of  this  contract  is  to  obtain  a  survey  of  the  scientific  and  technical  literature  on  models  of 
skill  fading  and  tools  for  determining  when  refresher  training  is  required. 

1.2  Canadian  Forces  Training  Philosophy 

The  December  1994  Defence  White  Paper  confirmed  the  need  for  Canada  to  maintain  multi-purpose, 
combat-capable  sea,  land  and  air  forces  to  protect  Canada  and  to  project  Canada’s  interests  and  values 
abroad.  In  order  to  maintain  these  multi-purpose  forces,  the  Canadian  Army,  Navy  and  Airforce 
require  "needs-trained"  personnel. 

In  peacetime,  training  is  the  most  important  activity  for  the  Canadian  Forces  (Atkinson,  1995),  where 
training  focuses  on  providing  general  purpose  combat  forces  that  allow  Canada  to  fight  and  win  in 
battle  if  necessary.  It  is  also  the  view  of  the  Canadian  Forces  that  training  to  develop  a  general 
purpose  combat  capability  meets  the  need  established  by  the  United  Nations  (UN)  for  military 
peacekeeping.  Thus,  it  is  the  training  philosophy  of  the  Canadian  Forces  that  the  best  training  for  the 
spectrum  of  conflict  (conventional  warfare  to  peacekeeping)  pertains  to  basic  skills  specific  to 
classification. 

Standards  drive  training  to  ensure  that  military  personnel  are  physically  fit,  capable,  and  can  form  the 
basis  of  effective  units.  Standards  provide  a  measurable  reference  by  which  the  CF  can  compare, 
evaluate,  and  identify  deficiencies.  For  the  Land  Force,  the  Army  Systems  Approach  to  Training 
(ASAT)  establishes  the  means  by  which  it  can  measure  performance,  correct  deficiencies,  and 
maximize  efficiencies. 

1.2.1  Military  Approach  to  Training 

It  is  necessary  for  all  personnel  to  be  trained  in  the  basic  military  skills  before  advancing  to  collective 
and  combined  training.  If  a  deficiency  is  identified  during  initial  evaluation,  the  individual  must 
rectify  that  deficiency  before  he/she  can  operate  effectively  as  part  of  a  unit.  Commanders  are  tasked 
to  ensure  intensive  refresher  training  is  conducted  to  ensure  the  soldier  quickly  reach  the  required 
standard.  Should  someone  fail  to  reach  the  standards  following  this  intensive  refresher  training  period, 
more  detailed  training  should  be  initiated. 

1.2.2  Training  Cycle-General 

Training  in  the  CF  is  progressively  planned  and  can  be  divided  into  basic,  individual  and  collective 
categories. 
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1. 2.2.1  Basic  Training 

The  basic  training  portion  is  covered  by  recruit  courses  and  gives  military  personnel  grounding  in 
basic  military  skills  to  prepare  them  for  the  more  advanced  training.  During  basic  training  army 
soldiers  are  taught  to  never  lose  sight  of  the  fact  that  their  basic  function  is  to  be  ready  at  short  notice 
to  engage  an  enemy.  Since  all  military  personnel  must  be  prepared  to  defend  themselves,  it  follows 
that  regardless  of  their  classification  or  occupation,  all  must  reach  a  reasonable  standard  of  individual 
combat  effectiveness. 


Figure  1.1:  Recruit  Firing  the  C7A1  Rifle 
1. 2.2.2  Individual  Training 

Military  personnel  must  master  basic  military  skills,  built  upon  those  learned  during  recruit  training, 
before  they  can  become  successful  members  of  a  team,  whether  or  not  they  are  in  a  combat  unit. 
Individual  training  comprises  three  parts: 

a.  General  military  training  (GMT), 

b.  Leadership  training,  and 

c.  Technical/trades  training. 

The  individual  training  phase  is  designed  to  mould  individuals  for  the  first  specific  position  they  will 
assume  in  a  unit.  In  addition  to  GMT  and  trades  training,  there  is  a  program  to  gradually  develop  the 
soldier's  professional  and  personal  qualities,  which  include  courage,  a  sense  of  responsibility, 
endurance,  alertness,  inquisitiveness,  discipline,  loyalty,  and  leadership.  On  completion  of  the 
individual  phase  of  training  military  personnel  will  still  not  be  fully  trained.  They  will,  however,  have 
learned  a  number  of  important  military  skills  and  are  ready  to  join  their  unit  as  a  member  of  a  crew, 
section,  or  team. 
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Figure  1.2:  Infantryman  Firing  the  C9LMG 
1. 2.2.3  Collective  Training 

Collective  training  ultimately  includes  the  grouping  of  all  the  combat,  combat  support,  combat  service 
support,  air  and  other  services  on  a  formation  level  exercise.  It  is  the  culmination  of  the  training  cycle. 
Collective  training  in  principle  covers  all  units  in  the  formation  and  must  be  progressive,  starting  with 
the  sub-sub-unit  and  continuing  upwards  to  unit  and  formation  exercises.  Under  these  conditions, 
operational  training  can  be  closely  supervised  and  it  will  follow  a  logical  progression.  Several  stages 
of  training  can  however,  be  done  concurrently.  Training  exercises  must  replicate,  as  closely  as 
possible,  real  battle  conditions. 

1.2.3  Land  Force  Training  Standards 

Personnel  receive  their  training  from  three  areas:  from  Canadian  Forces  (CF)  or  environmental 
courses;  from  annual  refresher  training;  and  for  those  participating  in  peacekeeping  operations,  pre¬ 
deployment  training.  The  Land  Force’s  general  purpose  combat  military  training  is  based  a  series  of 
standards: 

a)  Individual  Battle  Task  Standards  (IBTS) 

b)  Combined  Arms  Battle  Task  Standards 

•  Brigade  Group  Battle  Task  Standards 

•  Combat  Team  Battle  Task  Standards 

•  Corps  Battle  Task  Standards 

All  military  personnel  in  the  Land  Force  train  to  the  Individual  Battle  Task  Standards.  This  ensures  a 
common  standard  for  personnel  of  all  trades  or  environments  while  employed  on  Land  Force  Bases 
and  Units.  Brigade  Group  and  Combat  Team  Battle  Task  Standards  insure  that  units  and  formations 
can  perform  to  a  common  standard.  Corps  Battle  task  Standards  focus  on  specific  Corps  tasks,  e.g. 
infantry  C7A1  rifle  marksmanship.  Every  unit  in  the  Land  Force  conducts  annual  refresher  training 
and  tests  each  soldier,  sailor,  airman  or  airwomen  posted  in  its  establishment  on  the  Individual  Battle 
Task  Standards.  Additionally  all  land  units  train  and  are  evaluated  against  their  Corps,  Battle  Group 
and  Combat  Team  Battle  task  Standards. 
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Figure  1.3:  Leopard  Cl  Crews  During  Armour  Corps  Gunnery  Evaluation 

Achievement  of  the  Battle  Task  Standards  are  required  prior  to  deploying  on  operations.  However,  it  is 
not  possible  to  maintain  all  Army  personnel  at  this  level  of  operational  preparedness.  Some  units  will 
require  higher  levels  of  preparedness  than  others  based  on  assigned  tasks,  missions,  and  roles.  It  is  a 
Commander’s  responsibility  to  ensure  that  resources  are  available  to  these  units  to  meet  these  higher 
levels  of  preparedness.  These  same  resources  will  also  determine  the  level  of  preparedness  that  can  be 
reasonably  expected  of  certain  units  and  individuals.  It  is  understood  that  given  limitations  in 
ammunition,  time,  or  other  resources,  certain  Battle  Task  Standards  will  not  be  met  by  all  units  and 
personnel.  There  remains,  however,  a  requirement  to  ensure  the  principle  components  of  these  Battle 
Tasks  are  evaluated  so  as  to  provide  a  foundation  on  which  training  can  be  built,  in  order  to  meet  the 
standards.  The  IBTS  programme  provides  this  foundation  and  will  involve  yearly  evaluation  by  means 
of  a  Tests  of  Elementary  Training  (TsOET)  for  the  Battle  Tasks.  Successful  completion  of  the  TsOETs 
will  be  necessary  for  personnel  reporting  for  pre-deployment  training. 

The  three  principles  governing  the  ASAT  are  performance,  systems  approach,  and  maximum 
efficiency.  Using  these  principles,  the  following  approach  to  Battle  Task  evaluation  should  be 
followed: 

a)  Performance.  All  tasks  will  be  performance-oriented  to  a  logical  conclusion 
(application).  The  evaluation  of  all  tasks  will  be  oriented  to  the  operational  requirement. 

b)  System  Approach.  All  tasks  will  be  constantly  monitored  and  compared  to  operational, 
doctrine,  and  national  goals  and  their  effectiveness  evaluated.  Feedback  through 
operational  reporting  and  training  validation  will  be  conducted. 

c)  Maximum  Efficiency.  There  is  no  denying  the  fiscal  realities  of  peace.  Training  is  a 
costly  exercise  to  rectify  deficiencies.  If  there  is  no  deficiency,  there  is  no  need  to  expend 
limited  resources  and  time  correcting  something  that  is  not  deficient.  Therefore,  all  tasks 
in  this  regulation  should  be  tested  prior  to  initiating  refresher  training.  If  a  deficiency  is 
identified  (i.e.  The  standard  is  not  achieved),  only  then  should  a  soldier  receive  refresher 
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training  to  correct  the  deficiency.  Correct  targeting  of  high  failure  risk  populations  is 
required  to  ensure  max  efficiency  is  attained. 

1.2.4  Peacetime  Restraints  Affecting  Training 

A  number  of  restraints  affect  the  quality  and  quantity  of  training  possible. 

1.2.4. 1  Budgetary  Considerations 

Budgetary  restraints  have  now  become  a  fact  of  life  for  all  military  trainers.  The  costs  pertinent  to 
equipment,  ammunition,  transportation  and  logistic  requirements  must  be  carefully  assessed  before 
any  training  is  undertaken. 


Figure  1.4:  Crew  Firing  the  ERYX  Anti-Tank  Missile.  Note 
Each  Training  Missile  Costs  $13,000 

1. 2.4.2  Organizational  Considerations 

In  peacetime,  war  establishments  are  usually  restricted.  Commanders  must  adapt  appropriately  when 
this  occurs.  For  example,  there  may  be  a  battery  missing  in  an  artillery  regiment,  a  squadron  in  an 
armoured  regiment,  or  a  service  support  unit  may  lack  a  particular  field  capability.  To  offset 
manpower  deficiencies,  the  employment  of  the  reserves  and  militia,  either  as  individuals  or  as  formed 
organizations,  is  always  be  considered  by  the  regular  force. 

1. 2.4.3  Equipment  Considerations 

Equipment  in  peacetime  is  not  always  available  and  often  has  to  be  pooled.  During  the  annual  training 
cycle  it  is  often  necessary  for  units  to  provide  equipment  for  demonstrations  or  for  other  formations 
which  were  not  included  in  the  original  training  program. 
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1. 2.4.4  Ammunition ,  Explosives  and  Pyrotechnics 

Emphasis  is  placed  on  the  creation  of  realism  to  stimulate  interest  on  all  training  exercises.  One  of  the 
many  characteristics  of  battle  is  noise.  Until  soldiers  are  accustomed  to  the  crack  and  whine  of  bullets 
and  shells,  it  is  difficult  for  them  to  think  calmly  under  that  stress.  The  extent  to  which  such  noise  can 
be  produced  in  training  depends  almost  entirely  upon  the  training  ammunition,  pyrotechnics,  and 
explosives  available.  Unfortunately  supplies  are  usually  small,  and  an  atmosphere  resembling  battle  is 
difficult  to  achieve.  Operational  commanders  are  therefore  required  to  economize  and  determine 
when  the  best  training  value  can  be  gained  from  the  limited  resources. 

7 .2.4.5  Transportation  and  Logistics  Considerations 

The  movement  of  military  equipment  is  expensive.  Thus,  it  may  be  desirable  to  move  a  few  priority 
items  of  equipment  to  an  exercise  area  and  arrange  the  loan  of  equipment  from  other  units  in  the 
vicinity. 


1. 2.4.6  Air  Considerations 

Air  transport  is  generally  in  short  supply  in  peacetime  and  demands  for  it  must  be  made  well  in 
advance  of  planned  exercises. 

7.2.4. 7  Training  Restrictions 

Training  is  also  subject  to  a  number  of  peacetime  limitations,  such  as  restrictions  on  digging,  cutting 
trees  for  camouflage,  movement  off  roads,  or  running  vehicles  at  night  without  lights. 
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2.  Scope  and  Goals 


2.1  Scope  of  the  Literature  Review 

Training  in  a  large  array  of  skills  creates  a  problem  for  a  professional  organization  such  as  the  CF. 
Because  forgetting  can  occur  over  any  time  period  without  practice  (e.g.,  Goldberg  &  O'Rourke,  1989; 
Hagman  &  Rose,  1983;  Schendel  &  Hagman,  1982),  the  skills  of  those  trained  will  gradually  degrade, 
eventually  to  the  point  at  which  skills  do  not  meet  required  levels  of  proficiency.  This  is  true  for  any 
training  organization  and  many  others  have  considered  the  issues  surrounding  it  (e.g.,  Wisher,  Sabol, 
&  Ellis,  1999): 

•  How  quickly  does  forgetting  occur  for  different  kinds  of  skills? 

•  Are  some  individuals  more  likely  to  forget  than  others? 

•  What  instructional  strategies  are  effective  in  reducing  forgetting? 

•  How  difficult  will  it  be  to  reacquire  forgotten  skills? 

The  first  of  these  questions  is  central  and  this  literature  review  will  address  it  in  detail.  The  primary 
goal  of  this  literature  review  is  to  identify  and  review  models  for  predicting  skill  retention  that  can  be 
applied  in  the  military  domain.  A  secondary  goal  is  to  identify  the  fundamental  factors  that  affect  skill 
retention  and  will  affect  the  applicability  of  models  of  skill  retention. 

A  literature  review  examining  skill  fading  must  consider  a  variety  of  skills.  In  particular,  we  will 
systematically  examine  factors  affecting  training  and  skill  retention  with  respect  to  the  different  kinds 
of  skills  relevant  to  the  CF.  Techniques  that  promote  good  retention  for  one  type  of  skill  may  not  be 
suitable  for  another  type  of  skill.  Consequently,  it  is  important  to  determine  a  framework  for 
interpreting  the  literature  in  a  way  meaningful  to  the  skill  requirements  of  the  CF. 

This  literature  review  is  not  intended  to  be  a  comprehensive  survey  of  the  theoretical  and  empirical 
research  surrounding  skill  acquisition  and  retention.  Rather,  it  will  seek  to  address  several  core  issues 
of  relevance  to  the  prediction  of  skill  retention. 

First,  the  literature  review  is  limited  to  questions  of  skill  retention.  For  this  reason,  we  will  not 
explore  skill  acquisition,  transfer  of  skills,  or  other  related  topics,  except  as  they  affect  skill  retention. 

Second,  the  review  will  focus  on  models  for  predicting  skill  retention.  The  goal  of  the  project  is  to 
determine  an  accurate  means  of  determining  when  refresher  training  will  be  needed  to  sustain  military 
tasks.  Consequently,  there  is  a  need  for  detailed  quantitative  models  of  skill  retention.  We  will  not 
ignore  qualitative  models  and  other  approaches  to  estimating  skill  retention,  but  will  emphasize  the 
potential  to  provide  predictions  usable  by  trainers  and  commanders  to  determine  refresher  training 
needs.  As  part  of  the  review  we  will  examine: 

•  The  type  of  skills  to  which  various  models  apply. 

•  Empirical  support  for  the  predictive  validity  of  models. 

•  The  information  required  by  models. 

•  The  practicality  of  use  of  models. 

•  The  cost  of  use  of  models. 

Although  the  focus  of  the  review  is  on  models  of  skill  retention,  we  will  conduct  a  thorough  review  of 
factors  that  affect  skill  retention.  Knowledge  of  these  factors  is  crucial  to  evaluation  of  existing 
models  and  development  of  future  models. 
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Finally,  the  literature  review  is  focused  on  skills  relevant  to  the  military  domain.  For  this  reason  we 
excluded  research  pertaining  to  sports  psychology.  Many  military  tasks  involve  motor  and  perceptual 
skills  but  a  large  number  are  highly  cognitive,  requiring  complex  decision  making  and  reasoning.  We 
will  discuss  literature  pertaining  to  retention  of  all  kinds  of  skills,  but  will  emphasize  research  directly 
examining  retention  of  skills  in  the  military  context. 


2.2  Approach 

Despite  the  broad  skill-base  of  the  CF,  the  focus  of  the  literature  review  must  be  on  those  skills  and 
training  factors  that  pertain  strongly  to  the  military.  Research  on  learning,  training,  and  skill  retention 
in  other  domains,  such  as  business  and  education,  can  be  relevant  but  must  be  interpreted  with  respect 
to  the  skill  requirements  of  the  CF.  Thus,  we  reviewed  the  literature  with  respect  to  the  basic, 
specialized,  and  command  skills  relevant  to  the  CF. 

Of  particular  relevance  to  the  CF  are  factors  specific  to  the  military  domain,  such  as  career  paths, 
posting  cycles,  and  so  on.  These  factors  affect  training  and  retention  but  are  unlikely  to  have  been 
considered  in  the  general  literature  on  skills.  Thus,  we  made  special  effort  to  survey  military  scientific 
research  to  determine  how  these  factors  might  affect  skill  retention. 

Our  review  has  not  concentrated  exclusively  on  any  single  service  but  addressed  issues  of  skill  retention 
common  to  the  Army,  Navy,  and  Air  Force.  Although  some  differences  in  skills  and  training  needs  likely 
exist  across  the  different  services,  there  are  many  commonalties,  so  that  lessons  learned  for  one  service 
are  applicable  for  the  others.  To  explore  the  results  of  the  literature  review,  we  developed  three  practical 
examples  of  skill  learning  and  retention  directly  relevant  to  the  CF.  The  examples  illustrated  the  factors 
affecting  skill  retention  for  basic,  specialized,  and  command  skills  common  to  the  three  services.  This 
technique  was  used  to  highlight  practical  results  as  well  as  ensure  relevance.  The  examples  were 
developed  with  the  input  of  Subject  Matter  Experts  (SMEs)  from  the  CF. 

We  adopted  a  practical  focus  in  our  review  and  interpretation  of  the  scientific  literature.  The  ultimate 
goal  was  to  develop  training  design  recommendations  to  promote  skill  retention  for  the  CF.  We 
surveyed  the  applied  science  literature  to  identify  guidelines.  In  addition,  we  surveyed  the  basic 
science  literature  with  the  aim  of  identifying  theories  and  empirical  results  that  indicate  practical 
lessons  learned. 

We  also  adopted  a  focus  on  skill  retention.  Research  on  skill  acquisition,  knowledge  representation, 
and  similar  issues  was  relevant  to  the  literature  review  and  we  gave  that  due  consideration.  However, 
the  goal  of  the  review  was  to  identify  ways  for  determining  optimal  training  and  refresher  training 
programs.  Thus,  all  issues  were  reviewed  as  they  relate  to  the  issue  of  skill  retention. 

2.3  Method 

A  literature  review  is  a  systematic  study  and  organization  of  a  large  body  of  scientific  research.  The 
key  to  success,  then,  is  to  develop  a  framework  for  interpreting  the  literature  and  generating  clear  and 
useful  insights,  lessons  learned,  and  recommendations  for  future  research  and  practice.  We  followed  a 
number  of  steps  to  complete  the  literature  review. 

2.3.1  Develop  Systematization  Framework 

The  first  step  of  the  literature  review  is  to  develop  a  framework  for  organizing  and  interpreting 
empirical  results  and  theories  of  skill  retention.  The  framework  is  a  system  for  classifying  literature  in 
a  way  meaningful  to  the  CF.  Such  a  framework  is  needed  to  make  sense  of  the  literature  and  to  relate 
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results  to  the  specific  training  and  skill  retention  needs  of  the  CF.  The  framework  will  identify  the 
fundamental  factors  that  affect  skill  acquisition  and  retention  as  it  relates  to  the  military  domain.  It  is 
likely  that  different  combinations  of  these  factors  will  necessitate  different  approaches  to  training  to 
yield  optimal  skill  retention. 

We  were  able  to  identify  two  fundamental  factors  relevant  to  the  CF  (see  Table  1).  The  first  was  the 
type  of  skill  under  consideration.  This  factor  likely  exerts  a  great  deal  of  influence  over  retention  in 
itself.  In  addition,  however,  this  factor  almost  certainly  interacts  with  numerous  other  factors  (e.g., 
training  type,  frequency  of  refresher  training,  etc.)  to  determine  skill  retention.  Two  levels  of  type  of 
skill  are  commonly  distinguished  (e.g.,  Anderson,  1990);  perceptual/motor  and  cognitive.  The  former 
refers  to  tasks  that  predominantly  entail  physical  activities  and/or  basic  perceptual  activities,  such  as 
visual  monitoring.  The  latter  refer  to  the  very  broad  range  of  tasks  that  entail  decision  making, 
memory,  and  reasoning  activities 

The  second  factor  was  the  military  task  domain.  This  factor  plays  a  role  in  determining  the  type  of 
skills  required  as  well  as  a  number  of  factors  that  could  potentially  affect  skill  retention,  such  as 
complexity,  frequency  of  training  (due  to  availability  and  posting  cycle  issues),  and  available  training 
resources.  We  were  able  to  distinguish  at  least  three  levels  of  this  factor;  basic/general,  specialized, 
and  command.  Basic/general  tasks  are  those  that  are  required  or  trained  for  a  large  proportion  of  the 
CF,  whereas  specialized  tasks  are  required  by  a  relatively  small  proportion.  Specialized  tasks  are 
likely  to  be  more  complex  and  cognitive  than  basic  tasks.  Command  tasks  are  highly  cognitive  but  are 
distinguished  here  because  they  entail  a  distinctive  set  of  skills  drawn  from  a  wide  range  of  task 
domains  (Bryant  &  Webb,  1999). 

These  two  factors  served  as  a  preliminary  systematization  framework.  This  preliminary  framework 
was  evaluated  and  revised  on  the  basis  of  feedback  from  SMEs  and  results  of  the  literature  search. 

We  then  reviewed  the  literature  in  relation  to  the  framework.  Results  of  the  literature  review  were 
classified  with  respect  to  the  framework  so  that  we  could  relate  theories,  models,  and  lessons  learned 
to  the  goals  and  concerns  of  the  CF. 


Factor  ^rrjfef  ^ ^  W-  '■  ,§  ■? 

fy  /t*  ^<f> 

.’1 

Type  of  Skill 

Perceptual/Motor 

Cognitive 

Military  Task  Domain 

Basic/General 

Specialized 

Command 

Table  1.1  -  Fundamental  Factors  Affecting  Skill  Retention 


2.3.2  SME  Review 

The  goal  of  the  literature  was  to  identify  theories  and  lessons  learned  that  are  applicable  to  the  CF. 
Consequently,  an  important  step  was  to  review  military  skills  and  training  requirements  with  SMEs. 
The  purpose  of  this  review  was  to: 

•  Identify  critical  issues. 

•  Identify  examples  of  relevant  skills  and  training  to  be  used  to  illustrate  the  results  of  the 
literature  review. 
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•  Prioritize  the  focus  of  the  literature  and  limit  its  scope. 

•  Validate  the  systematization  framework. 

2.3.3  Search  Literature 

The  first  step  in  conducting  the  search  of  the  literature  was  to  identify  relevant  databases  of  scientific 
research.  We  surveyed  databases  for  psychology,  human  factors,  military,  professional/business,  and 
other  related  domains  to  compile  a  set  of  databases  to  search. 

The  next  step  was  to  generate  a  set  of  keywords  based  on  experience  and  SME  input.  Keywords  were 
organized  in  terms  of  the  validated  systematization  framework  to  yield  combinations  of  keywords  that 
focused  on  the  conceptual  distinctions  relevant  to  the  goals  of  the  literature  search. 

The  prioritized  set  of  keywords  was  applied  to  the  databases  to  identify  roughly  200  abstracts/titles. 
These  were  subjected  to  a  brief  review  to  establish  the  scope,  level  of  detail,  and  currency  of  their 
content.  Of  these,  we  selected  roughly  65  for  detailed  review  and  these  became  the  basis  of  the 
literature  review. 

2.3.4  Review  Selected  Articles 

Once  we  had  selected  the  key  articles,  we  conducted  a  thorough  review  of  each  article.  The  review: 

•  Identified  factors  and  issues  relevant  to  skill  retention. 

•  Identified  theories  and  models  of  skill  acquisition  and  skill  retention. 

•  Identified  models,  tools,  and  techniques  of  reducing  skill  fading. 

2.3.5  Generate  Survey  of  Literature  and  Lessons  Learned 

Based  on  the  review  of  key  articles,  we  generated  an  account  of  the  major  theories  and  models 
relevant  to  skill  retention.  The  review  allowed  us  to  evaluate  these  theories  based  on  scientific  validity 
and  applicability  to  the  CF.  Our  literature  survey: 

•  Developed  a  comprehensive  list  of  factors  promoting  skill  acquisition  and  skill  retention. 

•  Developed  a  comprehensive  list  of  tools  and  techniques  that  can  be  used  to  promote  skill 
retention. 

•  Classified  factors,  theories,  and  techniques  based  on  the  systematization  framework. 

The  literature  survey  provided  a  detailed  account  of  the  scientific  literature.  This  survey  was  focused 
on  the  issues  relevant  to  the  goals  of  the  CF  (as  developed  through  the  review  with  SMEs). 

The  final  step  of  the  literature  survey  was  to  generate  a  set  of  design  recommendations  for  military 
training.  These  recommendations  were  categorized  by  the  systematization  framework. 
Recommendations  advanced  practices  and  techniques  that  could  improve  the  quality  and  efficiency  of 
training  efforts  by  the  CF. 

2.4  Tasks 

As  part  of  the  project,  we  completed  the  following  specific  tasks: 

•  Start-up  meeting. 

•  Determine  and  prioritize  requirements. 

•  Review  focus  with  SA  and  SMEs. 

•  Frame  keyword  search. 
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•  Conduct  preliminary  search. 

•  Review  findings  of  preliminary  search  and  reframe. 

•  Conduct  full  search. 

•  Review  titles  and  prioritize. 

•  Create  an  EndNote  database  of  references. 

•  Obtain  articles  and  add  to  the  database. 

•  Review  articles. 

•  Draft  report. 

•  Review  draft  report  with  SA. 

•  Revise  and  submit  final  report. 


2.5  Acronyms 

AFOQT 

AFQT 

APS 

ANOVA 

ARI 

ASAT 

ASVAB 

BST 

CCS 

CF 

COFT 

d 

DK 

EOMM 

8 

GIL 

GMT 

IBTS 

IRR 

M 

MOOTW 

MOS 

NTC 

OOTW 

PC 

PGTS 

PKS 

PRIME 

RRS 

SAF 

SME 

TsOET 

TSQ 

UDA 

UN 


Air  Force  Officer  Qualifying  Test 

Armed  Forces  Qualification  Test 

Air  Force  Specialty 

Analysis  of  Variance 

Army  Research  Institute 

Army  Systems  Approach  to  Training 

Armed  Services  Vocational  Aptitude  Battery 

Basic  Skills  Trainer 

Combat  Service  Support 

CF 

Conduct-of-Fire  Trainer 
Effect  Size 

Declarative  Knowledge 
Ease-of-Movement  Matrix 
General  Cognitive  Ability 
Graphical  Instruction  in  LISP 
General  Military  Training 
Individual  Battle  Task  Standards 
Individual  Ready  Reserve 
Motivation 

Military  Operation  Other  Than  War 

Military  Occupational  Specialty 

National  Training  Center 

Operation  Other  Than  War 

Performance  Component 

Precision  Gunnery  Training  System 

Procedural  Knowledge  and  Skill 

Precision  Range  Integrated  Maneuver  Exercise 

Readiness  Reporting  System 

Semi-Autonomous  Forces 

Subject  Matter  Expert 

Tests  of  Elementary  Training 

Transfer  of  Skills  Questionnaire 

User's  Decision  Aid 

United  Nations 
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U.S.  United  States  of  America 

USAREUR  U.S.  Army  Europe 
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3.  Nature  of  Skills,  Tasks,  and  Training 


3.1  Skills  and  Skill  Fading 

3.1 .1  Kinds  of  Skills 

Skill  can  be  defined  in  two  ways.  First,  it  is  the  achievement  of  some  criterion  level  of  performance  at 
a  task.  Thus,  a  person  who  can  perform  a  task  is  said  to  possess  some  level  of  skill  and  a  person  who 
performs  the  task  well  is  said  to  possess  expertise,  or  an  extremely  high  level  of  skill.  In  practical 
terms,  this  definition  is  often  of  interest  in  the  military  because  it  describes  capabilities  of  personnel. 

In  terms  of  understanding  skill  retention,  however,  a  second  definition  is  needed.  Skill  is  any  kind  of 
learned  procedure,  problem  solving  technique,  or  sequence  of  steps,  although  researchers  often 
concentrate  on  learned  behaviors  that  depend  upon  motor  processes  (Schendel  &  Hagman,  1991). 

Thus,  we  can  understand  skills  in  terms  of  their  underlying  cognitive  and  motor  representation  and 
processes.  We  distinguish  skills,  which  are  acquired  through  formal  training  or  experiential  learning, 
from  non-leamed  abilities  or  basic  human  capabilities. 

Skills  are  goal-directed  and  organized  in  complex  sets  of  skills  that  are  used  together  in  a  work  domain 
(e.g.,  Kraiger,  Ford,  &  Salas,  1993).  Thus,  skills  exist  as  meaningful  cognitive  structures  or  networks 
that  guide  behavior. 

Just  as  there  is  a  huge  variety  of  tasks  or  jobs  that  people  perform,  there  are  a  wide  variety  of  skills 
that  can  be  acquired.  These  range  from  motor  and  perceptual  skills  to  cognitive  skills  to  interpersonal 
and  management  skills.  Training  leads  to  acquisition  of  procedures,  knowledge,  and  motivation  or 
affective  components  (Kraiger  et  al.,  1993).  We  should  not  focus  exclusively  on  procedures  because 
they  require  both  information,  in  the  form  of  knowledge  and  cognitive  processes  acquired  to  perform 
procedures,  and  attitudinal  and  motivational  dispositions  that  support  performance.  Simply  put,  one 
cannot  effectively  perform  a  task  without  requisite  knowledge  or  the  will  to  do  it. 

Skills  also  rest  upon  basic  information  processing  and  decision  making  capabilities  (Wisher,  Sabol,  & 
Ellis,  1999).  Many  military  tasks,  such  as  trouble-shooting  and  tactical  command,  involve  working 
with  large  amounts  of  information  in  an  ill-defined  problem  space.  The  procedures  to  follow  in  these 
cases  are  less  well  defined  and  necessarily  involve  many  decision  points. 

3.1.2  Stages  of  Skill  Development 

A  widely  accepted  theory  of  the  development  of  expertise  is  Anderson’s  three-stage  theory  (Anderson, 
1995).  This  theory  describes  the  transitions  in  knowledge  and  skill  level  from  novice  to  expert.  It  is  a 
general  model,  meant  to  apply  to  cognitive  and  behavioural  domains.  Thus,  it  is,  in  principle, 
applicable  to  any  kind  of  military  task  but  should  be  validated  for  any  specific  application. 

According  to  Anderson’s  model  (See  Anderson,  1995,  Ch.  5),  the  first  stage  of  skill  acquisition  is  the 
cognitive  stage.  At  this  stage,  the  individual  develops  declarative  knowledge  of  the  task,  including  its 
rules,  procedures,  constraints,  and  so  on.  All  of  this  knowledge  is  factual  and  explicitly  stored  in  the 
declarative  memory  store  (Tulving,  1983).  To  access  this  knowledge,  the  individual  consciously 
recalls  facts  about  the  task  and  attempts  to  use  them  to  guide  performance.  Thus,  at  this  stage,  there  is 
little  procedural  knowledge  and  the  individual  must  devote  a  great  deal  of  mental  effort  to  directing  his 
or  her  actions. 
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The  second  stage  is  the  associative  stage.  In  this  stage,  the  individual  gradually  notices  and  corrects 
errors  in  his  or  her  declarative  knowledge  and  strengthens  the  associations  between  cues  and  problem 
elements  and  the  actions  or  steps  needed  to  perform  the  task.  Thus,  the  individual  forms  greater 
associative  memory  (e.g.,  if-then  rules)  that  can  be  used  to  guide  behaviour.  Performance  is  still  not 
based  on  true  procedural  knowledge  but  the  stored  associations  allow  for  faster,  less  effortful 
performance. 

The  final  stage  is  the  autonomous  stage.  Here,  the  associative  productions  developed  earlier  become 
automatic.  Knowledge  is  stored  in  procedural  memory  that  can  be  accessed  implicitly  and  without 
conscious  effort  (or  very  little).  Performance  is  initiated  in  response  to  situational  cues  that  determine 
which  learned  procedures  should  be  called  upon. 

This  theory  is  consistent  with  observations  of  increased  automaticity  and  proceduralism  with  expertise 
in  a  variety  of  domains  (Anderson  &  Fincham,  1994).  However,  to  fully  understand  expertise  in 
cognitive  domains  (those  requiring  decision  making  and  reasoning),  we  must  identify  differences  in 
the  knowledge  structures  of  experts  and  novices. 

Experts  have  more  comprehensive  and  better  organized  domain  knowledge  than  novices  (Lesgold, 
Rubinson,  Feltovich,  Glaser,  Klopfer  &  Wang,  1988;  Sternberg,  1996;  Wiley,  1998).  This  is  almost  a 
tautology  but  it  indicates  two  important  processes  of  expertise  acquisition,  namely  a)  the  identification 
of  crucial  concepts,  cues,  and  procedures  in  the  domain,  and  b)  the  organization  of  knowledge  to 
facilitate  rapid  and  accurate  access  to  those  concepts,  cues,  and  procedures.  Thus,  as  a  person 
develops  expertise,  memory  becomes  specialized  to  recognize  and  retrieve  appropriate  responses  to 
problems  in  that  domain. 

In  addition  to  proceduralization,  a  process  of  schematization  also  occurs  as  one  develops  expertise. 
The  expert  creates  mental  frameworks  that  describe  the  kinds  of  problems  encountered  in  the  domain, 
the  kinds  of  cues  that  indicate  problems,  and  the  kinds  of  solutions  that  can  be  applied.  Anderson 
(1995)  distinguishes  between  tactical  and  strategic  learning.  Tactical  learning  deals  with  the 
acquisition  of  skills  and  procedures,  following  the  three  stages  of  Anderson’s  model.  Strategic 
learning  refers  to  learning  how  to  organize  one’s  problem  solving.  Creating  mental  schemata  to 
organize  domain  knowledge  allows  one  to  determine  the  best  means  to  solve  problems  and  quickly 
judge  the  applicability  of  solutions. 

Anderson’s  theory  implies  that  training  should  focus  more  on  procedures  than  information.  Because 
expertise  involves  acquiring  sets  of  specific  procedures,  training  can  be  enhanced  by  analyzing  what 
those  procedures  are.  Thus,  a  prerequisite  to  training  is  a  thorough  task  analysis.  The  results  of  this 
analysis  will  be  specific  tasks  and  procedures  that  can  be  explicitly  taught  to  novices.  Such  training 
will  be  especially  effective  if  tasks  are  broken  into  components,  each  of  which  can  be  mastered  more 
easily  than  the  whole  task  in  its  entirety. 

3.1.3  Defining  Skill  Retention 

Before  examining  skill  fading,  we  must  clarify  the  concepts  of  skill  retention  and  loss.  In  one  sense, 
skill  fading  (or  reduction  of  skill  retention)  is  the  measurable  decrement  in  performance  of  a  skill 
relative  to  a  criterion.  Ultimately,  we  are  concerned  with  the  levels  of  performance  of  tasks,  on  which 
this  definition  focuses.  This  definition,  however,  is  simply  the  observable  symptom  of  internal 
processes.  Of  greater  concern  is  the  maintenance  or  sustainment  of  skills  as  learned  behaviors  and 
procedures  over  long  periods  of  time  without  practice  (Schendel  &  Hagman,  1991). 

There  are,  however,  two  general  potential  causes  of  performance  degradation  over  time.  Reduced 
performance  may  reflect  some  degradation  of  the  perceptual,  motor,  and  cognitive  processes  that 
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underlie  skilled  performance.  This  could  take  the  form  of  decay  or  break-down  of  the  learned 
procedures.  Reduced  performance,  however,  could  also  reflect  the  inability  to  perform  learned  skills. 

In  this  case,  the  perceptual,  motor,  and  cognitive  processes  of  the  learned  skill  may  be  completely 
intact  but,  for  some  reason,  not  usable  by  the  individual. 

This  distinction  is  based  on  the  long-standing  distinction  between  availability  and  accessibility  of 
human  memory  (Tulving  &  Pearlstone,  1966;  Tulving,  1983).  Observation  of  some  decrement  in 
performance  does  not  necessarily  indicate  a  change  in  the  availability  of  a  skill.  Rather,  it  only 
indicates  some  loss  of  accessibility.  Reduced  accessibility  may  result  from  degradation  of  the  learned 
skill  but  it  may  reflect  factors  or  processes  related  to  the  retrieval  and  execution  of  the  skill.  This 
distinction  is  crucial  because  it  has  clear  implications  for  the  prediction  and  remediation  of  skill  loss. 

3.1 .4  Relation  of  Acquisition  and  Retention 

It  is  also  important  to  consider  the  goals  of  training  when  examining  retention.  Typically,  the  main 
goal  of  training  is  to  promote  long-term  performance  in  the  real  world.  Much  training,  however, 
focuses  on  achieving  high  levels  of  performance  in  the  training  course.  Training  is  the  only 
opportunity  that  trainers  have  to  affect  and  assess  performance  but  performance  in  the  training  session 
is  not  itself  the  goal.  An  implicit  assumption  of  most  training  courses  is  that  procedures  that  enhance 
performance  during  training  necessarily  enhance  long-term  performance  in  the  real  world  (Schmidt  & 
Bjork,  1992). 

This  assumption,  however,  is  not  always  true,  as  illustrated  by  experiments  that  demonstrate 
manipulations  that  maximize  performance  during  training  but  lead  to  inferior  long-term  performance,  or 
vice  versa.  Landauer  &  Bjork  (1978),  for  example,  observed  that  massed  practice  of  a  name-learning 
task  yielded  better  performance  during  training  than  did  spaced  practice  but  worse  performance  on  tests 
of  long-term  retention.  In  fact,  introducing  difficulties  for  learners  in  the  form  of  variability  in  practice 
and  feedback  can  enhance  later  performance  (Schmidt  &  Bjork,  1993). 

Acquisition  and  retention  do  not  appear  to  be  separable  phenomena  (Schmidt  &  Bjork,  1993).  Given 
the  goal  of  enhancing  long-term  performance,  acquisition  can  only  be  measured  in  terms  of  retention. 
That  is  because  performance  during  training  is  an  imperfect  predictor  of  long-term  performance. 

Given  the  distinction  between  availability  and  accessibility,  it  is  clear  that  good  acquisition  itself  does 
not  guarantee  later  performance.  Transfer  of  skills  learned  in  training  to  real  world  tasks  depends  on 
many  factors  outside  the  training  environment,  including  individual  and  organizational  issues 
(Kavanagh,  Lance,  O’Brien,  Stennett,  McMillen,  &  Solomonson,  1997). 

3.1 .5  Empirical  Studies  of  Skill  Retention 

Before  considering  models  for  predicting  skill  retention,  it  is  worth  reviewing  several  basic  empirical 
findings  concerning  the  nature  of  skill  retention.  Decrements  in  performance  can  be  described  by  a 
power  function  relating  level  of  performance  to  the  retention  interval  between  training  and  test  (e.g., 
Anderson,  1995;  Wixted  &  Ebbesen,  199 1)1: 

d’  =  A.Tb 

d’  =  measure  of  performance 
A  =  asymptote  of  curve 
b  =  slope  of  function 


1  The  power  function  is  often  expressed  in  logarithmic  form  as  Log(d’)  =  A  —  b  log  T. 
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T  =  time 


This  function,  illustrated  in  Figure  3.1,  is  negatively  accelerated,  meaning  that  performance  declines 
most  rapidly  soon  after  learning  then  at  an  increasingly  slow  rate  over  time.  The  two  parameters  by 
which  skill  loss  can  be  characterized  are  the  slope  of  the  curve,  describing  the  rate  of  skill  loss,  and  the 
asymptote,  which  describes  the  level  at  which  performance  levels  out.  Wixted  and  Ebbesen  (1991) 
have  observed  that  the  power  function  offers  the  most  accurate  description  of  retention  for  a  range  of 
tasks  and  materials  (see  also  Grant  &  Logan,  1993). 


Retention  Interval  (Time) 

Figure  3.1  -  Idealized  Power  Function  Relating  Retention  Performance  to  Retention  Interval 

The  nature  of  the  skill  loss  function  indicates  that  one  can  expect  proportionately  large  degrees  of  skill 
loss  after  only  moderate  amounts  of  time.  This  has  been  confirmed  in  numerous  studies  examining 
retention  of  military  tasks.  In  one  example,  Wisher,  Sabol,  Sukenik,  and  Kern  (1991)  examined  skill 
loss  by  twenty  thousand  Individual  Ready  Reserve  (IRR)  soldiers  mobilized  by  the  U.S.  Army  for 
Operation  Desert  Storm.  These  soldiers  had  completed  active  service  and  were  pretrained  and 
experienced  but  had  not  been  on  active  duty  for  some  time.  They  had  received  no  training  while  in  the 
IRR.  As  expected,  Wisher  et  al.  (1991)  found  that  soldiers  from  IRR  call-up  had  markedly  lower 
levels  of  skill  performance  and  knowledge  in  a  number  of  Military  Occupational  Specialties  (MOS) 
than  continuously  active  soldiers.  Similarly,  Burcham  (1998)  found  that  retention  of  Tactical 
Unmanned  Vehicle  tasks  by  soldiers  showed  rapid  decline  from  two  to  twelve  weeks,  with  the  most 
severe  decriment  in  performance  coming  at  two  weeks. 

Another  implication  of  the  skill  loss  function  is  that  different  skills  should  exhibit  different  rates  of 
skill  loss  and  different  asymptote  levels  of  performance.  Childs  and  Spears  (1986),  for  example, 
found  that  some  pilot  skills  decayed  more  rapidly  than  others.  Wisher  et  al.  (1991)  distinguished  three 
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components  of  most  tasks:  knowledge,  decision,  and  execution.  These  components,  however,  exhibit 
different  decay  rates  when  assessed  independently.  Research  indicates  that  memory  for  job 
knowledge  depends  on  how  that  knowledge  is  measured  but  generally  shows  fairly  rapid  forgetting 
(Wisher,  Sabol,  &  Ellis,  1999).  Research  examining  retention  of  decision  skills  suggests  more 
moderate  rates  of  decay  (Wisher  et  al.,  1999).  In  contrast,  continuous  skills,  which  encompass  many 
highly  practiced  motor  routines  (e.g.,  typing),  exhibit  little  or  no  loss  over  long  retention  intervals 
(Arthur,  Bennett,  Stanush,  &  McNelly,  1998;  Childs  &  Spears,  1986).  Depending  on  the  task,  military 
personnel  may  be  required  to  acquire  all  three  kinds  of  skills,  which  will  then  degrade  at  different 
rates. 

To  complicate  the  problem  of  predicting  retention,  numerous  factors  other  than  the  type  of  skill  also 
affect  the  nature  of  the  skill  loss  function.  We  will  discuss  these  factors  in  detail  in  Section  3.1  but 
factors  such  as  the  type  of  training,  degree  of  initial  learning,  and  individual  differences  affect  the 
slope  and  asymptote  of  the  function.  Thus,  to  predict  retention,  one  must  consider  a  multidimensional 
representation  of  the  skill. 

Although  decay  of  skill  performance  is  ubiquitous,  it  may  be  possible  for  learned  skills  to  endure  for 
very  long  periods  of  time.  A  number  of  researchers  have  proposed  a  “permastore”  memory  that 
preserves  skills  and  knowledge  with  essentially  no  amount  of  forgetting  (e.g.,  Fendrich,  Healy, 
Meiskey,  Crutcher,  Little,  &  Bourne,  1988).  Some  researchers  have  demonstrated  very  high  retention 
over  long  intervals  for  such  tasks  as  Spanish  language  training  (Bahrick,  1984,  cited  in  Fendrich  et  al., 
1988),  concept  learning  (Conway,  Cohen,  &  Stannope,  1991),  and  some  motor  tasks.  A  common 
explanation  for  this  high  level  of  retention  is  that  the  skill  has  become  automatized,  consuming  few 
cognitive  resources  and  seemingly  isolated  from  processes  that  lead  to  degradation  of  skills  and 
knowledge  (Anderson,  1995;  Fendrich  et  al.,  1988) 

3.2  Military  Tasks 

Military  tasks  require  a  variety  of  skills.  While  one  task  may  involve  predominantly  psycho-motor 
skills  i.e.  tracking,  another  may  involve  perceptual  skills  i.e.  identification  of  military  vehicles,  while 
still  others  require  cognitive  skills  i.e.  performing  a  combat  estimate  (problem  solving).  Most  military 
tasks  involve  a  number  of  skill  components. 

The  various  military  tasks  performed  in  the  Land  force  are  detailed  in  the  Battle  Task  Standard  (BTS). 
The  BTS  have  been  laid  out  and  numbered  by  combat  function  and  operation  of  war  or  transitional 
phase.  Battle  tasks  are  detailed  for: 
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a)  individual  (only  used  in  IBTS); 

b)  section/crew/detachment; 

c)  platoon/troop; 

d)  subunit; 

e)  combat  team; 

f)  battalion/regiment/battle  group;  and 

g)  brigade  group. 


3.2.1  Individual  Battle  Tasks 


The  IBTS  is  derived  from  common  Land  environment  specifications  and  are  common  to  all  members 
of  Land  Force  Command  exclusive  of  occupation  specific  skills.  They  are  taught  as  initial  individual 
training  through  general  military  training,  environment,  occupation  and  specialty  training.  Once  these 
Battle  Tasks  have  been  mastered  there  is  a  requirement  to  evaluate  proficiency  to  ensure  combat 
readiness.  The  Individual  Battle  Tasks  include: 


a)  Fire  the  C7/C8 

b)  Throw  Grenades 

c)  Fire  the  C9  Light  Machine  Gun 

d)  Fire  the  C6  General  Purpose  Machine  Gun 

e)  Fire  the  SRAAW  (L) 

f)  Fire  the  84  mm  Carl  G  SRAAW  (H) 

g)  Achieve  Army  Fitness 

h)  Apply  Military  First  Aid 

i)  Perform  Individual  Fieldcraft 

j)  Perform  Nuclear  Biological  Chemical  Defence 

k)  Apply  Mine  Awareness 

l)  Navigate  Using  a  Map  and  Compass 

m)  Communicate  Using  Communications  Equipment 

n)  Identify  Armoured  Fighting  Vehicles  and  Aircraft 


Please  see  Annex  A  for  an  example  of  an  Individual  Battle  Task 

3.2.2  Occupation  Specific  Battle  Tasks 

While  all  land  Force  personnel  must  be  able  to  master  the  IBTS,  they  must  also  master  their  trade  or 
occupation  specific  Battle  Tasks.  These  specific  battle  tasks  are  detailed  in  the  following  standards: 

a)  Air  defence  artillery  BTS 

b)  Armour  BTS 

c)  Field  artillery  BTS 

d)  Combat  service  support  BTS 

e)  Engineer  BTS 

f)  Infantry  BTS 

g)  Medical  BTS 

h)  Signal  BTS 

Please  see  Annex  B  for  an  example  of  an  Infantry  Battle  Task. 
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3.2.3  Collective  Battle  Tasks 


Individuals  do  not  operate  alone,  but  perform  their  missions  in  close  cooperation  with  other  arms, 
fighting  as  part  of  brigade  groups,  battle  groups,  combat  teams,  and  company  groups.  Consequently, 
Battle  Group  and  Combat  Team  Battle  Task  Standards  (BTS)  are  used  to  train  and  evaluate  the 
personnel  and  units  at  the  battalion  and  company  level  respectively.  Combat  Team  Battle  Tasks 
include  those  listed  in  Table  3.1. 

COMMAND 

1001,  Battle  Procedure 

1002,  Establish  and  Operate  a  CP 

FIREPOWER 

2001 ,  Employ  Indirect  Fire  Support 
INFORMATION  OPERATIONS 

3001 ,  Maintain  Operations  Security 

3002,  Screen 
MANOEUVRE 

4001,  Attack 

4002,  Establish  a  Fire  Base 

PROTECTION 

5001 ,  Cross  a  Contaminated  Area 
5012,  Emplace  Obstacles 

SUSTAINMENT 

6003,  Treat  and  Evacuate  Casualties 

6004,  Handle  PWs  /  Detainees 
Table  3.1.  Combat  Team  Battle  Tasks. 

Please  see  Annex  C  for  an  example  of  a  Battle  Group  Task. 


3.3  Military  Training 

3.3.1  Theories  of  Training 

Fallesen  and  Pound  (1998)  identify  three  broad  approaches  to  training.  One  is  a  formal  approach  that 
is  highly  structured  and  focuses  on  explicit  rules  and  procedures.  The  training  is  derived  from  formal 
theories  and  models  and  conveys  a  great  deal  of  declarative  knowledge.  Lecture -based  courses  are 
examples  of  this  approach. 

A  second  approach  is  based  on  intuitive  theories  of  decision  making  and  performance  (Bryant  & 
Webb,  1999).  This  approach  emphasizes  performance,  cue  differentiation,  and  the  acquisition  of 
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automaticity.  Training  is  done  by  practical  exercises  with  feedback  to  allow  individuals  to  acquire 
experience  comparable  to  being  on  the  job. 

The  third  approach  is  a  hybrid  approach  that  combines  explicit  teaching  of  concepts  and  procedures 
but  reinforces  this  with  practical  exercises.  The  first  step  is  to  identify  expert  strategies  for  solving 
problems  in  the  domain  and  teaching  novices  how  to  apply  these  strategies. 

Fallesen  and  Pound  (1998)  surveyed  the  literature  and  identified  over  65  individual  training  strategies 
(see  Pound  &  Fallesen,  1994,  cited  in  Fallesen  &  Pound,  1998).  They  found  that  informal  training 
strategies  are  often  preferred  over  formal  strategies  for  training  in  military  settings.  That  is,  military 
training  emphasizes  recognitional  and  procedural  techniques  rather  than  informational  content.  The 
goal  of  training  appears  to  be  to  help  trainees  develop  experience  that  can  be  applied  in  the  field, 
especially  the  ability  to  recognize  situations.  Thus,  current  training  techniques  are  generally  consistent 
with  intuitive  theories  of  decision  making. 

Nevertheless,  Fallesen  and  Pound  (1998)  advocate  a  hybrid  training  approach.  Instruction  includes 
components  of  declarative  description  of  concepts  and  procedures.  These  are  supported  by  exercises 
designed  to  reinforce  those  concepts  and  procedures.  Trainees  learn  by  applying  skills  in  realistic 
scenarios.  Thus,  this  approach  involves  more  implicit  learning  than  does  classroom  instruction  but 
could  be  helpful  early  in  training  when  novice  trainees  function  in  the  cognitive  stage  (see  Anderson, 
1995). 

Kozlowski  et  al.  (1998)  advocate  a  similar  approach,  involving  practical  exercises  designed  to  give 
trainees  experience  with  tasks.  To  facilitate  comprehension  and  retention,  exercises  start  at  a  simple 
level  and  become  more  complex  as  trainees  master  the  material.  Kozlowski  et  al.’s  (1998)  approach 
also  emphasizes  the  use  of  feedback  to  promote  learning.  In  particular,  they  advocate  the  use  of 
feedback  to  provide  evaluation,  attribution,  and  guidance. 

Evaluation  lets  the  trainee  know  whether  his  or  her  performance  was  correct  or  not.  This  is  essential 
information  but  not  sufficient  to  help  the  trainee  leam.  Feedback  that  provides  attribution  tells  the 
trainee  why  his  or  her  performance  was  incorrect.  Specific  attribution  provides  a  breakdown  of  the 
correct  and  incorrect  steps  taken  in  an  exercise  and  allows  trainees  to  modify  their  understanding  of 
the  task  at  a  detailed  level.  Finally,  guidance  provides  information  on  how  to  correct  errors  that 
trainees  can  incorporate  in  their  understanding. 

3.3.2  Training  Outcomes 

One  goal  of  training  is  to  develop  expertise  in  a  given  skill  or  set  of  skills.  Some  research  has 
attempted  to  characterize  the  nature  of  expertise.  Experts,  of  course,  are  individuals  who  have 
performed  in  the  domain  longer  than  novices  and  experts  exhibit  higher  levels  of  performance.  The 
crucial  issue  concerns  the  cognitive  structures  that  experts  have  acquired  through  practice  that  lead  to 
the  better  performance.  Thus,  researchers  examining  performance  in  domains  ranging  from  chess  (de 
Groot,  1965)  to  solving  physics  problems  (e.g.,  Glaser  &  Chi,  1988,  cited  in  Anderson,  1995)  have 
attempted  to  identify  the  specific  ways  in  which  experts  differ  from  novices. 

Some  of  the  essential  elements  of  expertise  (see  Federico,  1995)  are  listed  in  Table  3.2.  Many  of  these 
elements  are  basically  descriptive.  Experts  perform  faster  and  that  is  a  feature  by  which  we  recognize 
expertise.  Other  elements  are  difficult  to  interpret.  Experts  possess  better  short-term  and  long-term 
memory  skills  but  one  can  ask  whether  this  indicates  that  memory  improves  with  practice  or  that 
superior  memory  skills  allow  individuals  to  become  experts.  Still  other  elements  indicate  a  degree  of 
refining  or  improvement  in  certain  cognitive  skills.  Thus,  experts  exhibit  better  organization  of 
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knowledge  so  that  they  are  able  to  mentally  group  related  concepts  and  create  integrated 
representations  of  problems. 

itoitwnto  of  Expwrtli|j-:^,^,.^ 


Experts  perceive  great  numbers  of  interpretable  patterns  in  their  subject  matter,  indicating 
excellent  organization  of  knowledge 

Experts  perform  faster  and  with  fewer  errors  than  novices 

Expert  possess  superior  short-term  memory  (STM)  and  long-term  memory  (LTM)  for  domain¬ 
relevant  material 

Experts  perceive  and  represent  problems  in  domain  at  a  deeper,  more  conceptual,  level  than 
novices 

Experts  quantitatively  analyze  problems  to  represent  them  mentally  and  specify  situations  and 
constraints 

Experts  possess  substantial  self-monitoring  skills  when  detecting  errors,  checking  solutions 
Experts  spend  proportionately  more  time  than  novices  constructing  problem  representations 
Experts  possess  larger  sets  of  schemata  than  novices 

Expert  knowledge  is  organized  to  greater  degree  so  it  is  more  accessible,  functional,  efficient  to 
use 

Experts’  problem  perception  is  schema-driven,  whereas  novices’  problem  perception  is  based 
on  general  search  strategies 

Experts  exhibit  more  context-dependent  performance  than  novices;  novices  possess  context- 
free  features  and  facts  and  rules  for  behaving 


Table  3.2  -  Elements  of  Expertise  (from  Federico,  1995) 


Perhaps  the  major  element  of  expertise  is  the  acquisition  of  knowledge.  Simply  by  practice,  the  expert 
encounters  many  situations  and  attempts  many  solutions  to  problems.  In  the  process,  this  information 
is  recorded  in  memory  and  organized.  This  store  of  information  is  extremely  valuable  for  decision 
making  components  of  skills. 

The  organization  of  knowledge,  of  course,  is  another  key  factor.  In  particular,  expertise  is  developed 
when  a  person  relates  domain  knowledge  in  terms  of  conceptual  rather  than  surface  elements.  Experts 
recognize  more  meaningful  aspects  of  problems  and  assign  greater  weight  to  the  conceptual  structure 
of  the  situation  than  novices  (Federico,  1995).  That  is,  experts  look  for  features  related  to  functions, 
goals,  intent,  and  so  on,  and  use  these  features  to  identify  the  problem  as  well  as  retrieve  appropriate 
solutions.  Novices  tend  to  look  for  the  most  salient,  observable  features  that  indicate  some  surface 
similarity  to  previous  experience.  The  greater  use  of  conceptual  organization  by  experts  allows  them 
to  identify  solutions  that  address  the  meaningful  issues  of  the  problem. 

It  is  important  to  note  that  experts  and  novices  do  not  necessarily  differ  in  the  amount  of  conceptual  or 
structural  data  they  use  when  solving  problems  (Federico,  1995).  Often,  experts  and  novices  sample 
the  same  data  or  information  when  performing  tasks.  Differences  in  their  performance  result  from 
how  experts  and  novices  use  data.  Novices  tend  to  perform  in  a  context-free  manner;  i.e.,  they  focus 
almost  exclusively  on  elements  of  the  problem  at  hand  and  ignore  important  situational  cues.  That  is, 
they  fail  to  connect  the  current  situation  to  previous  experience.  Experts,  on  the  other  hand,  perform 
in  a  context-dependent  manner;  i.e.,  they  rely  on  cues  that  help  them  categorize  the  type  of  problem 
and  relate  it  to  previous  experience.  Experts  may  not  process  the  data  at  hand  to  any  greater  degree 
than  novices  but  they  are  able  to  link  it  quickly  to  their  domain  knowledge. 
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3.3.3  Training  Implications  of  Expertise 

The  main  implication  of  research  on  expertise  is  that  training  should  provide  large  amounts  of 
practice.  Expertise  involves  acquiring  large  numbers  of  specific  problem  templates  and  solutions 
(Anderson,  1990).  Because  the  expert  ultimately  stores  this  knowledge  in  a  procedural  form,  it  is 
essential  that  people  directly  experience  domain  problems  by  doing  actual  tasks.  In  this  way,  they  can 
learn  about  all  aspects  of  the  domain  while  learning  how  to  implement  solutions. 

Of  particular  importance  in  training  is  linking  actions  and  solutions  to  schemata  (Lipshitz  &  Shaul, 
1997).  During  the  second  stage  of  expertise,  a  person  forms  associative  memories  that  allow  him  or  her 
to  identify  the  appropriate  actions  given  the  situation.  Explicit  training  can  facilitate  this  process  by 
identifying  the  critical  cues  and  factors  for  categorizing  situations  and  matching  them  to  a  template. 

A  second  implication  is  that  practice  should  be  concrete  and  realistic  but  should  also  highlight  the 
conceptual  structure  and  principles  of  the  domain.  The  conceptual  structure  consists  of  the  critical 
cues  and  associations  that  link  actions  to  templates  and  forms  the  basis  of  the  way  an  expert  identifies 
situations.  Thus,  training  should  help  people  look  past  the  perceptual  surface  structure  of  the  problem 
(i.e.,  the  raw  data)  and  identify  meaningful  aspects  and  relationships  in  the  situation. 

A  third  implication  is  that  trainees  need  different  forms  of  training  at  different  stages  of  expertise. 
Initially,  the  trainee  will  function  at  the  cognitive  level.  Training  must,  to  some  extent,  focus  on 
providing  declarative  knowledge  of  the  domain  in  an  analytic  format  that  the  trainee  can  understand 
and  incorporate  in  his  or  her  growing  understanding  of  the  domain.  The  training,  however,  must  also 
help  the  trainee  move  to  the  next  stage  by  providing  practice  in  categorizing  situations  and  associating 
responses.  At  the  second  stage,  declarative  information  becomes  less  valuable  and  the  focus  of 
training  should  be  on  expanding  the  associative  knowledge  base  of  trainees.  This  can  be  done  through 
extensive  practice  and  explicit  training  in  critical  cues.  Finally,  as  the  trainee  moves  into  the 
procedural  stage,  the  trainee  will  require  less  formal  instruction  other  than  to  find  and  remediate  errors. 
What  will  be  crucial  is  realistic  simulation  and  practice  to  help  the  trainee  apply  what  has  been  learned 
in  real  settings. 

Anderson  (1990)  advocates  a  componential  approach  to  training,  in  which  material  is  broken  into 
related  sets  of  elements.  These  elements  are  taught  individually  through  mastery  learning  techniques. 
With  mastery  learning,  trainees  study  one  component  at  a  time,  while  their  learning  is  monitored. 
Trainees  do  not  advance  until  they  have  mastered  the  component. 
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4.  Predicting  Military  Task  Retention 


Before  reviewing  models  for  predicting  skill  retention,  we  begin  by  examining  factors  that  affect  skill 
retention  with  the  aim  of  setting  the  groundwork  for  a  systematization  framework.  Such  a  framework 
is  needed  to  evaluate  models  and  understand  the  challenges  of  accurately  predicting  the  decay  of  skill. 
We  will  then  examine  several  modeling  approaches,  subjective,  qualitative,  and  quantitative. 

4.1  Factors  Affecting  Skill  Retention 

Researchers  have  spent  considerable  time  identifying  the  relevant  factors  that  affect  skill  retention 
(e.g.,  Arthur  et  al.,  1998;  Goldberg  &  O’Rourke,  1989).  With  these  factors  identified,  researchers  can 
characterize  their  relation  to  skill  retention  and  estimate  the  magnitude  of  their  effects.  This  is  the  first 
step  in  developing  a  useful  model  for  predicting  skill  retention  but  not  the  only  one.  Functions 
describing  the  relationships  between  factors  and  retention  must  be  scaled  to  indicate  the  course  of  skill 
loss  over  real  units  of  time. 

Much  of  the  previous  research  has  examined  the  qualitative  effects  of  factors;  that  is,  they  have  sought 
to  determine  the  valence  of  the  factor’s  effect  (positive  or  negative)  but  not  to  generate  a  function 
relating  levels  of  the  factor  to  levels  of  skill  retention.  This  is,  for  the  most  part,  a  practical  decision 
because  determining  the  quantitative  relation  requires  greater  expenditure  of  resources  in  terms  of 
time,  equipment,  test  subjects,  and  so  on.  Among  the  factors  that  have  been  examined  are  (see  e.g., 
Arthur  et  al.,  1998;  Elliott  &  Wisher,  1993;  Hagman  &  Rose,  1983): 

•  Length  of  retention  interval. 

•  Degree  of  overlearning. 

•  Task  characteristics. 

•  Methods  of  testing  for  original  learning  and  retention. 

•  Conditions  of  retrieval. 

•  Use  of  mnemonics  and  other  task  aids. 

•  Instructional  strategies  or  training  methods. 

•  Prior  experience. 

•  Individual  differences. 

These  factors,  and  others,  can  be  divided  into  four  basic  categories  (see  e.g.,  Driskell,  Willis,  & 
Cooper,  1992;  Elliott  &  Wisher,  1993;  Hagman  &  Rose,  1983;  Schendel  &  Hagman,  1991): 

•  Task  factors,  which  pertain  to  the  nature  of  the  task  or  skill  to  be  acquired;  e.g., 
complexity. 

•  Training  factors,  which  pertain  to  the  types  of  training  manipulations  employed  in  initial 
learning;  e.g.,  length,  style. 

•  Learner  factors,  which  pertain  to  the  individual  differences  and  factors  that  affect 
learning  and  retention;  e.g.,  aptitude,  experience. 


Humaroysre/iw  Incorporated 


Skill  Fading  Literature  Review 


Page  23 


•  Retention  interval  factors,  which  pertain  to  the  events  and  manipulations  occurring 
between  training  and  performance;  e.g.,  length,  opportunity  for  practice.2 

This  is  a  pragmatic  classification  scheme  and  there  is  no  theoretical  reason  to  expect  all  factors  within 
a  category  to  be  equally  predictive  of  skill  retention.  Nor  is  there  any  theoretical  reason  to  expect 
certain  categories  to  be  more  predictive  overall  than  another.  It  is  an  empirical  issue  as  to  the  strength 
of  relation  of  any  specific  factor  to  skill  retention.  Indeed,  research  suggests  that  some  factors,  such  as 
duration  of  training  and  training  workload,  have  little  effect  on  long-term  retention  in  and  of 
themselves  (Goldberg  &  O’Rourke,  1989). 

As  a  result,  we  will  not  examine  research  pertaining  to  all  factors  that  have  been  proposed  to  affect 
skill  retention.  Instead,  we  concentrate  on  those  factors  identified  as  the  most  relevant  and  used  most 
often  as  a  basis  for  prediction. 

4.1 .1  Retention  Interval 

Perhaps  the  most  obvious  factor  to  consider  is  the  retention  interval.  Most  people  intuitively 
understand  that  the  longer  the  period  between  training  and  performing  a  skill,  the  more  likely  a  person 
is  to  show  some  degree  of  forgetting  or  decrement  in  performance.  Indeed,  research  indicates  that  the 
course  of  skill  loss  is  highly  predictable;  performance  tends  to  decrease  rapidly  soon  after  training  and 
continues  to  drop  but  at  an  increasingly  slower  rate  as  the  retention  level  increases  (see  Section  2.1.5). 
The  problem  with  considering  retention  interval  as  a  factor  is  that  time  itself  is  presumably  not  a 
causal  factor  but  a  covariate  of  underlying  processes  (such  as  neurological  processes)  that  produce 
skill  loss  (Arthur  et  al.,  1998).  Both  memory  trace  decay  and  interference  between  learned  skills  have 
been  advanced  as  theoretical  accounts  of  the  effect  of  retention  interval,  although  neither  provides  a 
complete  explanation  (Adams,  1987). 

If  time  is  not  a  causal  factor,  the  amount  of  skill  loss  must  be  influenced  by  other  factors,  including 
characteristics  of  the  task,  training,  and  individual.  For  example,  researchers  have  suggested  that  the 
relation  between  skill  loss  and  retention  interval  is  mediated  by  the  degree  of  initial  learning,  method 
of  testing,  conditions  of  retrieval,  instructional  strategies,  among  other  factors  (Arthur  et  al.,  1998). 
Certainly,  retention  interval  itself  does  not  produce  skill  loss  because  some  skills,  such  as  highly 
practiced,  continuous  motor  skills,  show  very  little  decrement  even  over  long  retention  intervals 
(Arthur  et  al.,  1998). 

Nevertheless,  retention  interval  is  likely  to  covary  with  many  factors  that  do  have  a  causal  relation 
with  skill  loss.  This  fact,  in  conjunction  with  the  ease  of  measuring  retention  interval  relative  to 
measuring  other  factors,  make  retention  interval  a  convenient  variable  to  consider  in  predicting  skill 
retention.  The  CF  regularly  utilizes  retention  interval  for  identifying  when  to  impose  refresher 
training.  Historically,  many  of  the  battle  tasks  are  practiced  and  tested  annually  (Annual  Refresher 
Training).  As  well,  there  are  a  number  of  specialty  trades  training  policies  that  recommend  the 
frequency  and  progression  of  training  (see  Table  4.1). 


2  Researchers  often  identify  aspects  of  the  retention  interval  as  training  factors  but  it  is  useful  to  distinguish 
factors  associated  with  the  kind  and  quality  of  training  from  factors  associated  with  events  and  activities 
occurring  between  training  and  performance  of  tasks. 
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TYPE  OF  TRAINING 

LEVEL  OF  TRAINING 

DESCRIPTION 

DETACHMENT 

SECTION 

PLATOON 

Indoor  Tracking 

W 

W  =  weekly 

M  =  monthly 

Q  =  quarterly 

S  =  semi-annually 

A  =  annually 

Outdoor  Tracking 

M 

Battle  Run-Detachment 

Q 

Battle  Run-Section 

Q 

Platoon  Exercises 

S 

live  Fire 

A 

AFV  Identification 

W 

Gun  Drills 

M 

Fire  Control 

M 

Q 

TGT  Engagement  Data 

Q 

Table  4.1 :  Recommended  refresher  training  practices  for  the  TOW  anti-armour  system. 

4.1.2  Opportunity  to  Practice 

Another  factor  retention  interval  factor  is  the  opportunity  to  practice  afforded  individuals  between 
training  and  performance  of  the  skill.  One  source  of  evidence  supporting  this  factor  comes  from 
studies  of  the  effects  of  Operations  Other  Than  War  (OOTW)  on  retention  of  combat  skills.  OOTWs 
consist  of  humanitarian,  peace,  and  noncombatant  evacuation  operations  that  generally  do  not  exercise 
combat  skills  and  provide  almost  no  opportunity  to  practice  such  skills  (Blankmeyer,  1998). 

In  one  study,  Landry  and  Campbell  (1997)  hypothesized  that  OOTW  would  degrade  combat  readiness 
of  US  Army  units  and  that  the  effects  would  be  largely  predictable,  although  the  effects  should  vary  by 
type  of  unit  and  OOTW.  Fifty-seven  active  duty  Army  officers  with  OOTW  experience  responded  to 
a  questionnaire  by  rating  the  level  of  readiness  of  the  respondent’s  unit  before,  during,  and  after  the 
OOTW  deployment.  Respondents  indicated  that  prior  to  deployment  units  were  ready  on  almost  all 
normal  combat  skills.  Only  15%  of  respondents,  however,  reported  having  training  opportunities  for 
combat  skills  during  the  OOTW  deployment  and  most  respondents  indicated  that  the  impact  of  this 
was  to  degrade  combat  readiness.  Respondents  indicated  that  units  were  not  combat  ready  upon  return 
from  deployment  and  required  substantial  retraining  time. 

Although  most  respondents  in  Landry  and  Campbell’s  (1997)  survey  indicated  that  OOTW 
deployment  a  negative  effect  on  combat  readiness,  some  claimed  that  OOTW  improved  the  readiness 
of  the  unit.  Most  of  these  respondents,  however,  were  from  Combat  Service  Support  (CSS)  units, 
which  engage  in  fewer  combat  tasks  than  other  units.  Respondents  from  CSS  units  reported  that  their 
OOTW  tasks  (e.g.,  driving  trucks)  complemented  critical  “go-to-war”  tasks  to  some  degree.  Thus, 
these  units  seem  to  have  received  practice  during  the  OOTW. 

The  decline  in  combat  readiness  of  units  deployed  on  OOTW  suggests  that  lack  of  practice  promotes 
skills  loss  but  there  are  other  factors  that  could  confound  this  interpretation  (e.g.,  interference).  To 
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examine  the  role  of  practice.  Ford,  Quinones,  Sego,  &  Sorra  (1992)  defined  opportunity  to  perform  a 
skill  in  terms  of  three  dimensions: 

•  Breadth:  number  of  trained  tasks  that  a  trainee  actually  performs  on  the  job. 

•  Activity  level:  amount  of  task  repetition. 

•  Type  of  tasks  performed:  complexity  and  difficulty  of  tasks,  which  affect 
opportunity  to  perform. 

Together,  these  dimensions  define  the  amount  of  practice  a  trainee  will  receive  for  trained  skills.  Ford 
et  al.  (1992)  identified  several  factors  that  affect  opportunity  to  perform.  The  organization  plays  a 
large  role  as  each  department  or  functional  unit  creates  its  own  goals  and  values  that  guide  assignment 
of  personnel  to  tasks.  The  work  context  created  by  supervisor  and  peer  attitudes  toward  the  trainee 
will  also  increase  or  decrease  opportunities  to  practice  skills.  Finally,  individual  characteristics,  such 
as  level  of  ability,  motivation,  and  self-efficacy,  guide  performance  on  the  job. 

To  examine  the  impact  of  these  factors.  Ford  et  al.  (1992)  sent  surveys  to  personnel  trained  in  the  US 
Air  Force  Aerospace  Ground  Equipment  Airman  Basic-in-Residence  technical  training  course  and 
their  job  supervisors.  The  surveys  measured  opportunity  to  perform  trained  tasks  in  terms  of  breadth, 
activity  level,  and  types  of  tasks.  Correlation  analyses  indicated  that  work  context,  individual  factors, 
and  organization  were  each  significantly  related  to  opportnuity  to  perform  trained  tasks  on  the  job. 
Work  context  and  individual  factors  were  somewhat  more  strongly  related  to  opportunity  to  perform 
than  organizational  structure,  indicating  that  individuals  in  the  same  unit  may  have  different  levels  of 
practice  over  time. 

Other  studies  have  observed  a  direct  relation  between  the  length  of  the  non-utilization  period  and 
amount  of  skill  loss  (see  Hurlock  &  Montague,  1982,  for  a  review).  This  relation  holds  true  when  the 
interval  is  spent  without  practice.  It  is  also  clear,  however,  that  other  factors  interact  with  non¬ 
utilization  of  skills  to  cause  performance  to  decline  at  different  rates.  Although  some  studies  have 
found  no  difference  in  the  effectiveness  of  massed  and  spaced  practice  schedules  (Elliott  &  Wisher, 
1993),  others  indicate  that  the  predictability  of  the  practice  schedule  can  affect  trainees'  performance. 
In  one  study,  for  example,  Shute  and  Gawlick  (1995)  examined  the  effects  of  different  practice 
schedules  on  learning  and  retention  of  problem  solving  strategies.  Participants  used  either  a  consistent 
practice  schedule  or  engaged  in  a  varied  number  of  practice  trials  for  each  problem  type.  Contrary  to 
prediction,  participants  who  received  abbreviated  practice  opportunities  exhibited  better  retention  than 
those  who  received  consistent  practice.  Shute  and  Gawlick  (1995)  argued  that  variable  practice 
induces  learners  to  invest  more  effort  into  learning  and  to  develop  more  efficient  problem  solving 
strategies  that  are  less  subject  to  forgetting  over  time.  Wisher,  Kern,  Sabol,  &  Farr  (1994)  report  that 
aptitude,  as  assessed  by  the  US  Army’s  Armed  Forces  Qualification  Test  (AFQT),  also  mediates  the 
negative  effect  of  nonutilization  on  skill  retention. 

4.1.3  Degree  of  Learning 

One  of  the  most  important  determinants  of  the  level  of  performance  of  a  skill  after  some  retention 
interval  is  the  initial  level  of  learning.  A  great  deal  of  research  has  examined  the  effect  of  this  factor, 
although  often  this  research  has  confounded  the  effects  of  degree  of  learning  with  those  of  the  amount 
of  training  (e.g.,  Lance,  Parisi,  Bennett,  Teachout,  Harville,  &  Welles,  1998).  Hagman  and  Rose 
(1983),  for  example,  reviewed  studies  conducted  by  the  U.S.  Army  Research  Institute  (ARI)  that 
examined  the  effect  of  amount  of  training/leaming.  In  one  study  (Hagman,  1980b,  cited  in  Hagman  & 
Rose,  1983),  trainees’  performance  on  an  electrical  repair  task  was  a  direct  function  of  the  number  of 
times  they  had  performed  the  task  during  training,  suggesting  that  those  who  learn  more  initially  will 
retain  more  later.  Retention  is  not  solely  a  function  of  practice.  In  a  review  of  the  literature,  Hurlock 
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and  Montague  (1982)  found  that  other  factors  that  enhance  initial  learning,  such  as  degree  of  aptitude, 
facilitate  retention  over  time. 

From  this  literature  comes  the  concept  of  overtraining  or  overlearning,  in  which  trainees  receive 
additional  training  beyond  that  needed  to  achieve  some  criterion  level  of  performance  (Schendel  & 
Hagman,  1980,  cited  in  Hagman  &  Rose,  1983).  For  example,  given  that  one  group  is  trained  by  a 
number  of  trials  (X  trials)  required  to  achieve  a  criterion  of  one  correct  performance  of  a  task, 
overlearning  would  entail  administering  twice  that  number  of  training  trials  (2X  trials).  Retention  of  a 
range  of  skills  has  been  demonstrated  to  be  better  given  some  degree  of  overlearning  (Goldberg  et  al., 
1981,  cited  in  Hagman  &  Rose,  1983;  Schendel  &  Hagman,  1980,  cited  in  Hagman  &  Rose,  1983). 

Overlearning  can  enhance  retention  by  a  number  of  mechanisms.  It  may  strengthen  the  associations 
between  stimulus  and  response,  thereby  decreasing  the  probability  of  forgetting  (Arthur  et  al.,  1998; 
Lance  et  al.,  1998).  If  so,  overlearning  would  also  encourage  automaticity  or  proceduralization,  which 
reduces  cognitive  demands  and  promotes  better  long-term  performance  (Anderson,  1983). 

Overlearning  by  its  nature  entails  more  practice  of  a  skill,  providing  greater  feedback  regarding 
correctness  of  responses  (Arthur  et  al.,  1998;  Lance  et  al.,  1998). 

In  a  recent  study,  Lance  et  al.  (1998)  examined  the  effect  of  overlearning  on  retention  of  representative 
Air  Force  Specialties  (AFSs).  Airmen  were  categorized  on  the  basis  of  their  level  of  prior  experience, 
the  length  of  time  since  they  last  performed  the  relevant  tasks,  and  the  number  of  times  they  have 
performed  the  tasks  on  the  job.  Lance  et  al.  (1998)  measured  task  performance  for  all  airmen  and  found 
that  it  was  positively  related  to  prior  experience  and  the  number  of  times  tasks  were  performed.  The 
results,  however,  were  somewhat  mixed  as  this  relation  was  not  found  for  all  AFSs 

Driskell,  Willis,  and  Cooper  (1992)  suggest  that  there  is  some  limit  to  the  benefits  of  overlearning. 

They  performed  a  meta-analysis  of  research  to  determine  the  precise  effects  of  overlearning  and  found 
that  a  minimum  of  50%  overlearning  (i.e.,  training  plus  the  addition  of  trials  equal  to  one  half  the 
original  training)  was  required  to  observe  any  positive  effect  of  overlearning.  Thus,  if  too  little 
overlearning  is  attempted,  one  is  unlikely  to  observe  a  positive  relation  between  overlearning  and 
retention. 

Refresher  training  is  often  used  to  counteract  the  effects  of  forgetting  and  maintain  proficiency  of 
skills  (Schendel  &  Hagman,  1991).  The  key,  as  evidenced  by  the  focus  of  this  literature  review,  is  to 
determine  the  rate  at  which  skills  decay  and  schedule  refresher  training  such  that  proficiency  does  not 
decline  below  a  criterion  level.  Extra  training  enhances  retention  regardless  of  whether  it  is  provided 
during  the  initial  training  or  in  a  refresher  training  session  at  some  point  during  the  retention  interval 
(Schendel  &  Hagman,  1980,  cited  in  Hagman  &  Rose,  1983).  Other  research  on  overlearning  suggests 
that  refresher  training  at  any  point  in  time  can  have  a  positive  effect  on  long-term  retention  as  long  as  a 
sufficient  degree  of  overlearning  is  achieved  (Driskell  et  al.,  1991).  Practical  issues  of  cost  and 
availability  of  personnel,  however,  require  that  refresher  training  be  conducted  in  an  efficient  manner. 
Schendel  and  Hagman  (1991)  argue  that  refresher  training  sessions  be  conducted  at  intervals  similar  to 
usage  intervals  experienced  on  the  job.  That  is,  if  personnel  perform  a  task  about  once  every  month  on 
the  job,  then  refresher  training  need  not  be  scheduled  more  frequently  than  that. 

4.1 .4  Method  of  T raining 

The  amount  of  initial  learning  is  obviously  related  to  the  amount  of  initial  training.  Thus,  this  factor 
has  also  been  identified  as  a  predictor  of  skill  retention  (e.g.,  Hurlock  &  Montague,  1982).  The 
relationship  between  training  and  retention,  however,  is  not  so  simple  and  many  aspects  of  training 
affect  skill  retention.  Again,  Hagman  &  Rose  (1983)  have  summarized  early  research  and  noted  that 
studies  have  indicated  that  retention  performance  is  affected  by  such  factors  as  whether  trainees  are 
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tested  during  training,  whether  practice  trials  are  spaced  or  massed  during  training,  and  whether 
trainees  practice  on  constant  or  varied  equipment. 

Hagman  (1980a,  1980b,  cited  in  Hagman  &  Rose,  1983)  and  Hurlock  and  Montague  (1982)  have 
found  that  trainees  who  were  tested  during  training  performed  better  at  retention  than  those  who 
received  only  presentation  training.  Test  taking  may  simulate  the  actual  performance  environment, 
improving  transfer  from  training  to  the  job.  Test  taking  can  also  stimulate  practice  and  development 
of  more  effective  performance  strategies  by  trainees. 

Schendel  and  Hagman  (1980,  cited  in  Hagman  &  Rose,  1983)  and  Hagman  (1980c,  cited  in  Hagman  & 
Rose,  1983)  found  that  spacing  repetitions  and  practice  sessions  enhances  long-term  retention,  provided 
the  interval  between  repetitions  is  not  too  long  (see  Schendel  &  Hagman,  1991).  One  possible 
explanation  is  that  varied  practice  schedules  force  trainees  to  expend  greater  cognitive  effort  because 
they  must  engage  in  more  retrieval  of  learned  material.  Shute  and  Gawlick  (1995)  tested  this  possibility 
by  contrasting  the  impact  of  an  abbreviated  practice  condition  to  an  extended  practice  condition  on 
trainees  in  a  flight  engineering  task.  In  the  abbreviated  condition,  trainees  received  only  one  quarter  the 
problems  as  the  extended  condition.  Perhaps  counter-intuitively,  trainees  who  received  abbreviated 
practice  had  better  retention  than  those  in  the  extended  practice  condition.  Shute  and  Gawlick  (1995) 
suggest  that  trainees  in  the  abbreviated  condition  devoted  greater  effort  during  training  to  compensate 
for  fewer  practice  trials.  In  one  experiment,  some  trainees  received  mixed  practice  scheduling 
(abbreviated  in  the  first  half  of  training  then  extended  for  the  second  half,  or  vice  versa).  These  trainees 
performed  best  of  all  on  tests  of  retention,  again  indicating  that  variable  practice  schedules  enhance 
retention.  Spacing  also  allows  learners  to  store  skills  in  relation  to  a  greater  variety  of  retrieval  cues 
(Schendel  &  Hagman,  1991),  improving  use  of  skills  on-the-job. 

The  kinds  of  feedback  provided  during  training  also  affect  retention,  depending  on  their  informational 
content.  Feedback  can  serve  a  motivating  purpose,  in  which  case  its  effects  may  not  extend  beyond 
the  training  session.  When  feedback  contains  information  about  the  magnitude  and  direction  of 
performance  errors,  however,  it  directs  the  learner  towards  ways  of  correcting  the  error  and  improving 
performance. 

People  generally  need  feedback,  or  “knowledge  of  results,”  to  correct  errors,  observe  and  use  cues 
associated  with  task  performance,  and  generate  effective  procedures  (Hurlock  &  Montague,  1982). 

This  effect  can  extend  to  long-term  retention.  In  one  review,  Schmidt  (1997)  examined  the  benefits  of 
informational  feedback  and  observed  that  feedback  has  two  opposing  effects  on  retention.  Feedback 
aids  learning,  leading  to  better  initial  learning  and  longer  retention,  but  also  builds  reliance  on  the 
feedback.  In  the  latter  case,  the  learner  suffers  performance  decrement  when  the  feedback  is  no  longer 
present,  which  impairs  retrieval  of  skills  on  the  job.  The  goal  of  training  should  be  to  make  use  of  the 
informational  content  of  feedback  but  to  avoid  presenting  feedback  so  frequently  or  in  such  a  regular 
sequence  that  the  learner  incorporates  feedback  in  their  mental  representation  of  the  task. 

Some  training  techniques  provide  means  for  learners  to  continue  practicing  skills  after  the  training 
session  has  ended.  Cognitive  training  consists  of  mental  rehearsal  strategies  in  which  a  learner 
mentally  simulates  the  step-by-step  performance  of  a  task  (Childs  &  Spears,  1986).  This  requires 
instruction  in  the  task  and  guidance  toward  the  mental  cues  needed  to  imagine  performing  the  task. 
After  training,  however,  the  learner  can  use  the  technique  whenever  desired  to  provide  practice. 
Empirical  evidence  supports  the  effectiveness  of  cognitive  training  in  maintaining  flight  skills  (see 
Childs  &  Spears,  1986),  which  is  especially  effective  for  practice  of  perceptual  and  cognitive  aspects 
of  tasks. 
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4.1 .5  Similarity  of  Training  and  Performance  Environments 

Similarity  of  the  retention  environment  to  the  original  learning  environment  promotes  retention 
(Arthur  et  al.,  1998).  This  finding  derives  from  the  Encoding  Specificity  Principle  of  memory,  which 
states  that  cues  to  retrieval  will  be  effective  if  and  only  if  encoded  at  the  time  of  learning  (Tulving, 
1983).  In  the  context  of  skill  retention,  this  implies  that  perceptual  and  cognitive  cues  are  needed  to 
retrieve  and  perform  learned  skills.  If  the  retention  environment  does  not  present  these  cues, 
performance  will  suffer. 

Healy,  Fendrich,  Crutcher,  Wittman,  Gesi,  Ericsson,  and  Bourne  (1992)  review  a  number  of  studies 
demonstrating  cases  of  both  remarkably  good  skill  retention  and  very  poor  retention.  On  this  basis, 
they  advanced  the  Procedural  Reinstatement  Proposal  that  the  retention  of  a  learned  skill  depends  on 
the  extent  to  which  learning  procedures  are  reinstated  at  the  time  of  retention.  In  this  case,  the  cues  to 
retrieval  are  the  very  procedures  originally  learned.  The  implication  of  this  proposal  is  that  to  retain 
skills  over  long  intervals,  one  must  ensure  that  procedures  used  when  learning  the  skill  are  reinstated 
later.  For  example,  if  training  employed  a  job  aid  or  checklist  to  aid  learners  in  sequencing  steps,  that 
aid  will  be  an  important  cue  needed  to  reinstate  the  skill  at  a  later  time. 

A  study  by  Marmie  and  Healy  (1995)  illustrates  this  effect.  They  examined  learning  and  retention  of 
tank  gunner  skills,  which  were  trained  using  the  TopGun  simulator  used  by  the  U.S.  Army.  Other 
theoretical  perspectives  predict  poor  retention  of  these  skills  due  to  the  complexity  and  cognitive 
nature  of  the  task  (Adams,  1987,  cited  in  Marmie  &  Healy,  1995;  Driskell  et  al.,  1992,  cited  in  Marmie 
&  Healy,  1995).  The  procedural  reinstatement  proposal,  in  contrast,  predicts  extremely  good  retention 
because  the  task  is  largely  procedural  and  the  procedures  are  reinstated  during  actual  performance.  In 
two  experiments,  trainees  showed  very  little  forgetting  over  retention  intervals  up  to  22  months, 
verifying  predictions  of  the  procedural  reinstatement  proposal.  These  results  also  rule  out  a  simple 
monotonic  relationship  between  task  complexity  and  retention. 

4.1.6  Type  of  Task 

Although  Marmie  and  Healy  (1995)  suggest  limits  on  the  effects  of  task  complexity,  many  researchers 
have  considered  task  characteristics  to  be  very  important  to  skill  retention.  The  classical  distinctions 
include  (Arthur  et  al.,  1998): 

•  Physical  vs.  cognitive. 

•  Continuous  vs.  discrete. 

•  Natural  vs.  artificial. 

•  Speed  (require  performance  in  specified  amount  of  time)  vs.  accuracy  (require 
performance  to  a  specified  level  of  accuracy). 

In  their  review  of  the  literature,  Arthur  et  al.  (1998)  found  that  skills  associated  with  cognitive  tasks, 
natural  tasks,  speed  tasks,  and  open-loop  tasks  tend  to  be  better  retained  than  skills  associated  with 
physical,  artificial,  accuracy,  and  closed-loop  tasks.  Hurlock  and  Montague  (1982)  also  report  better 
retention  of  open-loop  or  continuous  tasks.  A  great  deal  of  research  indicates  that  continuous  skills 
show  little  or  no  forgetting  over  retention  intervals  of  months  or  even  years  (Schendel  &  Hagman, 
1991).  Hurlock  and  Montague  (1982)  speculate  that  discrete,  procedural  tasks  are  more  poorly 
retained  because  of  the  large  number  of  discrete  steps  to  remember  and  the  even  more  difficult 
problem  of  remembering  the  precise  sequence  of  steps.  Consistent  with  this  idea  is  the  finding  that 
tasks  that  have  a  meaningful  organization  or  coherence  of  steps  tend  to  be  better  retained  (Hurlock  & 
Montague,  1982). 
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Complex  procedural  skills  are  particularly  susceptible  to  rapid  decay  (Schendel  &  Hagman,  1991). 
Forgetting  of  such  skills,  however,  is  moderated  by  the  organization  of  the  steps  making  up  the 
procedure  (Schendel  &  Hagman,  1991).  One  advantage  of  well -organized  tasks  may  be  that 
procedural  steps  act  as  natural  cues  for  each  other,  making  them  easier  to  remember.  Shields, 
Goldberg,  and  Dressel  (1979,  cited  in  Hagman  &  Rose,  1983)  found  that  soldiers  generally  forgot 
steps  of  common  soldier  tasks  that  were  not  cued  by  equipment  or  previous  steps  (e.g.,  safety  steps). 
Other  studies  have  confirmed  that  uncued  steps,  steps  judged  to  be  difficult,  steps  addressing  safety, 
and  steps  at  the  beginning  and  end  of  the  task  tend  to  be  the  most  difficult  to  remember  (Osborn  et  al., 
1979,  cited  in  Hagman  &  Rose,  1983). 

4.1.7  Method  of  Testing 

Different  measures  of  performance  can  indicate  different  levels  of  retention  (Arthur  et  al.,  1998).  Two 
common  tests,  for  example,  are  recall  and  recognition.  Recall  requires  the  learner  to  generate  learned 
material  or  skills,  often  in  response  to  only  a  few  cues.  Recognition  requires  the  learner  to  simply 
assess  the  familiarity  of  material  and  verify  its  accuracy.  As  a  result,  recognition  performance  tends  to 
be  better  than  recall  performance  across  a  wide  range  of  tasks  (Arthur  et  al.,  1998).  Research  has 
shown,  however,  that  recall  and  recognition  are  largely  independent  measures  of  memory  (see 
Tulving,  1983).  These  two  tests  can  assess  different  aspects  of  memory  and,  hence,  reveal  differences 
in  retention  of  selected  components  of  a  skill. 

4.1 .8  Individual  Differences 

Although  many  individual  characteristics  could  potentially  affect  skill  retention,  researchers  have 
focused  on  a  few  fundamental  ones,  of  which  the  most  notable  is  general  ability  or  aptitude.  A 
common  hypothesis  has  been  that  learners  of  lower  ability  forget  skills  more  rapidly  than  higher 
ability  learners  (Lance  et  al.,  1998).  Indeed,  a  number  of  studies  have  found  that,  when  training  time 
is  fixed,  higher  ability  personnel  learn  more  than  lower  ability  personnel  and  that  this  translates  into 
better  retention  (see  Hurlock  &  Montague,  1982).  Other  studies,  however,  have  yielded  more  mixed 
results  as  aptitude  moderated  retention  for  some  but  not  other  tasks  (Lance  et  al.,  1998). 

Even  when  ability  is  found  to  affect  skill  retention,  however,  it  is  often  unclear  whether  that  factor  is 
truly  causal.  Hurlock  and  Montague  (1982)  report  that  when  high  and  low  ability  groups  are  trained  to 
the  same  level  of  proficiency  on  a  task  (which  will  take  different  amounts  of  training),  no  differences 
in  retention  are  observed.  A  great  deal  of  research  indicates  that  individuals  of  different  ability  levels 
do  not  differ  in  their  rates  of  skill  loss  (see  Schendel  &  Hagman,  1991). 

Another  issue  concerns  the  precise  nature  of  ability,  which  can  be  defined  in  a  number  of  ways.  One 
view  is  that  ability  corresponds  to  general  cognitive  ability  (referred  to  as  g)  (Spearman,  1927).  Ree, 
Carretta,  and  Teachout  (1995)  reviewed  a  number  of  studies  that  documented  the  effectiveness  of  g  as 
a  predictor  of  job  performance  and  suggested  that  g  could  predict  on-the-job  performance  of  US  Air 
Force  pilots  after  their  53-week  training  course.  They  collected  pilots’  scores  on  the  verbal  and 
quantitative  subtests  of  the  Air  Force  Officer  Qualifying  Test  (AFOQT)  to  serve  as  measures  of  g.  Job 
performance  was  assessed  by  flying  work  samples  and  academic  grades.  Composing  a  measurement 
model,  Ree  et  al.  (1995)  determined  that  g  had  only  an  indirect  effect  on  job  performance;  g  positively 
affected  prior  job  knowledge  and  job  knowledge  acquired  through  training,  which  in  turn  affected  job 
performance. 

Another  approach  to  defining  ability  is  to  assess  multiple  specific  abilities  to  determine  how  each 
affects  performance.  In  one  study,  Earles  and  Ree  (1992)  examined  the  validity  of  the  ten  aptitude 
tests  of  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  as  predictors  of  training 
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performance.  They  found  variability  in  the  effectiveness  of  the  various  subtests,  with  Arithmetic 
Reasoning  and  Mathematics  Knowledge  being  the  best  individual  predictors.  Composite  scales 
created  by  combining  sets  of  subtests  could  be  even  more  effective  predictors  of  performance.  Similar 
multi-scale  aptitude  tests  have  been  developed  for  other  domains,  such  as  air  traffic  control,  and  have 
sometimes  been  found  to  have  limited  predictiveness  of  training  performance  (Ackerman  &  Kanfer, 
1996). 

Motivation  is  another  commonly  investigated  individual  factor  that  seems  to  have  an  effect  on  training 
performance,  although  its  effect  on  retention  has  been  less  well  established  (Hurlock  &  Montague, 
1982).  Motivation  may  increase  learning  during  training,  which  leads  to  better  performance  over  time. 
Lack  of  motivation  after  training,  however,  does  not  seem  to  increase  the  rate  of  skill  loss.  Related 
personality  factors,  including  self-efficacy  and  mastery  beliefs  may,  however,  affect  the  expression  of 
learned  skills  over  time  (Zazanis  &  Lappin,  1998).  Self-efficacy  is  the  internal  sense  of  competence 
pertaining  to  a  given  skill.  Low  self-efficacy  has  been  found  to  impair  performance.  Consequently, 
factors  that  reduce  a  person’s  sense  of  competence  over  the  span  of  a  retention  interval  may  depress 
performance  further. 

4.2  Transfer  and  Retention  of  Skills 

Before  considering  models  of  skill  retention,  we  must  clarify  the  relation  between  skill  retention  and 
skill  transfer.  Typically,  researchers  consider  skill  retention  to  refer  to  situations  in  which  skills  are 
applied  in  work  settings  anticipated  during  training.  In  contrast,  skill  transfer  is  taken  to  refer  to 
situations  where  skills  are  applied  to  tasks  and  situations  not  explicitly  anticipated  during  training. 

Any  trained  skill,  however,  will  be  applied  in  situations  that  differ  in  some  ways  from  the  training 
environment,  so  that  it  is  never  the  case  that  training  and  performance  tasks  are  exactly  the  same 
(Schmidt  &  Bjork,  1992).  Thus,  skill  retention  and  skill  transfer  appear  to  differ  in  degree  rather  than 
quality.  Training  objectives  should  always  be  to  maximize  performance  when  skills  are  transferred 
from  training  to  the  job  environment  (Kraiger  et  al.,  1993). 

This  view  of  retention  is  consistent  with  the  procedural  reinstatement  proposal  (Healy  et  al.,  1992). 
Because  effective  performance  of  a  skill  depends  on  reinstating  the  procedures  used  to  acquire  the 
skill,  we  can  see  that  it  is  possible  to  envision  cases  in  which  “retention”  as  it  is  typically  defined 
might  be  worse  than  “transfer”  as  it  is  typically  defined. 

4.3  Subjective  Approaches 

Perhaps  the  most  straightforward  means  to  predict  retention  and  estimate  the  need  for  refresher 
training  is  to  ask  the  learner  how  much  of  the  learned  skills  have  been  retained.  Such  a  technique,  of 
course,  may  not  be  very  accurate.  This  section  reviews  subjective  approaches  that  seek  valid  and 
reliable  means  to  use  learners’  self-knowledge  to  estimate  skill  retention. 

4.3.1  Meta-Cognition  and  Self-Assessment 

Perhaps  due  to  an  inherent  mistrust  of  self-assessments  of  learning,  relatively  few  studies  have 
examined  learners’  abilities  to  estimate  their  own  level  of  skill  retention  in  applied  settings.  However, 
meta-cognition,  the  cognitive  processes  used  to  monitor  one’s  own  thoughts  and  memory  processes, 
has  been  studied  quite  extensively  in  laboratory  settings.  Early  on,  it  became  apparent  that  people 
were  fairly  good  at  predicting  the  ease  of  learning  of  verbal  materials;  that  is,  they  could  accurately 
predict  which  items  were  likely  to  be  well  remembered  (e.g.,  Underwood,  1966,  cited  in  Zechmeister 
&  Nyberg,  1982).  Research  also  investigated  related  judgements  of  knowing,  which  consist  of  rating 
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the  likelihood  of  remembering  studied  items.  This  judgment  differs  from  estimating  ease  of  learning 
by  estimating  what  is  known  rather  than  what  will  be  known.  The  two  are  not  exactly  the  same  as  an 
item  rated  as  difficult  to  learn  might,  after  extensive  study,  be  rated  as  well  known.  Research 
indicated  that,  overall,  people  make  accurate  judgments  of  knowing  but  that  their  ratings  do  not  reflect 
the  effects  of  some  basic  and  powerful  factors,  such  as  serial  position  and  meaningfulness  (e.g., 
Arbuckle  &  Cuddy,  1969,  cited  in  Zechmeister  &  Nyberg,  1982). 

Self  assessments  have  been  used  to  examine  skill  acquisition  and  job  performance  in  domains  such  as 
the  health  professions  (e.g.,  Schwiebert  &  Davis,  1995).  In  a  review  of  the  literature,  Gordon  (1991) 
compared  performance  in  health  care  training  as  self-assessed  with  performance  as  assessed  by  experts 
or  objective  tests.  He  found  that  generally  self-assessments  of  performance  were  closely  related  to 
generalized  self-attributions  (e.g.,  self-efficacy)  and  minimally  influenced  by  external  feedback  (tests, 
grades,  faculty  assessments).  Although  self-assessment  appeared  to  be  only  marginally  accurate, 
Gordon  did  note  findings  that  suggested  that  instruction  in  making  self-assessments  could  improve  the 
validity  of  this  technique  for  predicting  performance  levels. 

One  study  of  military  skill  retention  examined  the  usefulness  of  self-assessments  as  predictors  of  skill 
retention.  Wisher  et  al.  (1991)  examined  skill  retention  of  soldiers  in  the  Individual  Ready  Reserve 
(IRR)  who  were  mobilized  for  Operation  Desert  Storm.  These  soldiers  had  completed  active  service 
but  received  no  training  or  practice  in  combat  skills  while  in  the  IRR.  Wisher  et  al.  (1991)  distributed 
questionnaires  to  soldiers  after  their  re-certification  training.  The  questionnaires  assessed  how  often 
soldiers  perform  relevant  skills  in  civilian  life,  how  many  MOS  skills  were  remembered  at  the  time  of 
call-up,  how  technically  prepared  the  soldier  was  after  training,  and  overall  confidence  in  capability  to 
perform.  Wisher  et  al.  (1991)  then  gathered  actual  performance  data  of  soldiers  during  their 
mobilization  and  compared  this  to  their  assessments  of  knowing  from  the  questionnaires.  They  found 
that  soldiers’  self-assessments  were  good  predictors  of  retention  of  MOS  skills  but  that  they  were 
somewhat  biased,  being  too  optimistic. 

Other  research  suggests  that  self-assessments  are  relatively  poor  predictors  of  skill  retention.  Schendel 
et  al.  (1984,  cited  in  Schendel  &  Hagman,  1991)  found  that  soldiers’  predictions  of  their  own  retention 
of  a  task  accounted  for  only  about  10%  of  the  variance  associated  with  task  performance  scores.  Most 
of  these  soldiers  overestimated  their  retention.  Schendel  and  Hagman  (1991)  consider  self-assessment 
a  potentially  useful  technique  considering  its  convenience  and  specificity  but  perceive  it  as  not 
currently  refined  enough  to  be  used  in  the  field. 

4.3.2  Refresher  Training  Estimates 

Even  if  military  personnel  are  able  to  assess  the  state  of  their  knowledge  and  skill  retention,  they  may 
not  be  able  to  use  that  information  to  accurately  determine  their  need  for  refresher  training.  Schendel 
and  Hagman  (1982)  hypothesized  that  people’s  good  metacognitive  abilities  should  allow  them  to 
directly  estimate  their  need  for  refresher  training  and  to  adjust  their  own  refresher  training  schedule. 

In  their  study,  they  examined  learning  of  the  general  assembly/disassembly  of  the  M60  machinegun. 
Soldiers  in  the  control  condition  were  trained  to  criterion  on  this  task,  tested  for  retention  after  an  eight 
week  retention  period,  then  retrained  to  criterion.  Soldiers  in  an  overtraining  group  were  trained  to 
criterion  plus  an  additional  100%  overtraining,  then  similarly  tested  and  retrained.  A  refresher  training 
group  received  the  same  amount  of  additional  training  as  the  overlearning  group  but  received  it 
midway  through  the  retention  interval.  All  groups  were  asked,  prior  to  the  retention  test  estimate  how 
much  of  the  skill  they  retained  and  how  many  retraining  trials  they  would  need  to  return  to  their 
criterion  level  of  proficiency.  The  overtraining  and  refresher  training  groups  exhibited  better 
retention,  as  expected,  but  all  groups  were  very  accurate  in  making  their  refresher  training  needs 
estimates.  Gross  overestimates  of  need  occurred  only  infrequently  and  73%  of  soldiers’  estimates 
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were  within  one  trial  of  being  correct.  Thus,  Schendel  and  Hagman  (1982)  concluded  that  it  is 
possible  to  accurately  predict  refresher  training  need  by  soldiers’  subjective  self-assessments. 

4.3.3  Applicability  and  Practicality  of  Subjective  Approaches 

The  evidence  in  support  of  subjective  approaches  tends  to  be  inconclusive.  Certainly,  empirical 
findings  amply  demonstrate  that  people  have  some  ability  to  assess  their  own  levels  of  retention  and 
need  for  refresher  training  (Schendel  &  Hagman,  1982;  Wisher  et  al.,  1991)  but  it  is  unclear  exactly 
how  accurate  are  these  self-assessments.  Indeed,  to  be  useful  in  a  military  setting,  a  model  must  be 
calibrated  to  the  kinds  of  tasks,  training  and  performance  environments,  and  retention  intervals  in  real 
job  settings.  Further  research  is  needed  to  determine  how  self-assessments  are  affected  by  these 
factors  before  a  practical  approach  can  be  developed.  In  particular,  attention  should  be  devoted  to 
examining  military  tasks,  as  only  a  few  studies  have  addressed  this  domain. 

Regardless  of  the  true  accuracy  of  self-assessment,  a  major  obstacle  to  employing  a  subjective  model 
will  be  the  perception  among  commanders  and  trainers  that  self-assessments  cannot  be  trusted. 
Certainly,  subjective  approaches  are  vulnerable  to  malingering  in  the  form  of  deliberately  inaccurate 
self-assessments,  although  various  techniques  can  be  used  to  guard  against  this  (Iverson,  Franzen,  & 
McCracken,  1991).  Commanders  and  trainers  may  be  reluctant  to  base  training  decisions  on  a  model 
that  does  not  entail  objective  assessment  of  factors  determining  retention.  This  mistrust  may  be 
partially  mitigated  by  education  about  subjective  approaches  but  is  unlikely  to  be  completely 
eliminated. 

A  subjective  approach  requires  little  in  the  way  of  materiel  because  self-assessments  can  be  collected 
by  paper-and-pencil  or  computerized  questionnaire.  The  main  resource  requirement  is  manpower  to 
administer  and  monitor  the  process  of  collecting  self-assessments.  This  requirement,  however,  can  be 
reduced  by  training  commanders  and  staffs  in  the  field  to  administer  and  score  questionnaires 
themselves.  Self-assessments,  of  course,  must  be  gathered  for  each  individual  soldier,  meaning  that  a 
large  amount  of  data  must  be  analysed  to  determine  the  retention  level  at  the  unit  level. 

4.4  Qualitative  Approaches 

The  ultimate  goal  for  prediction  of  skill  retention  is  to  generate  accurate,  quantitative  estimates  of  the 
rate  at  which  skills  degrade  over  time.  This  level  of  precision,  however,  can  be  very  difficult  to 
achieve.  Consequently,  some  researchers  have  developed  models  that  offer  only  quantitative 
predictions  of  skill  retention.  That  is,  the  models  indicate  categorically  how  skills  fade  over  time  in 
relation  to  certain  key  factors.  These  models,  as  they  stand  to  date,  do  not  solve  the  practical  problems 
of  predicting  when  proficiency  of  skills  will  decline  below  a  criterion  or  when  refresher  training  will 
be  needed  but  they  do  offer  approaches  that  may  be  developed  into  quantitative  models  in  the  future. 

4.4.1  Training  Criteria 

One  of  the  classes  of  factors  that  will  affect  skill  retention  is  the  nature  of  the  training;  certain  forms  of 
training  will  lead  to  longer  retention  than  others.  Kirkpatrick  (1959a,  1959b,  1960a,  1960b,  cited  in 
Alliger,  Tannenbaum,  Bennett,  Traver,  &  Shotland,  1997;  see  also  Arthur  et  al.,  1998)  developed  a 
classification  scheme  to  evaluate  training  effectiveness  with  an  eye  toward  predicting  learning  and 
retention.  The  scheme  contains  four  categories  (or  steps  in  Kirkpatrick’s  taxonomy)  of  measures  of 
effectiveness  of  training  outcomes: 

•  Step  1:  Reactions  (liking,  attitudes  toward  the  training). 

•  Step  2:  Learning  (knowledge  obtained). 
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•  Step  3:  Behavior  (using  learned  principles). 

•  Step  4:  Results  (desired  benefits  to  the  training  organization). 

Kirkpatrick’s  classification  scheme  allows  trainers  to  measure,  albeit  roughly,  the  characteristics  of  the 
training  course  to  determine  the  extent  to  which  learners  will  benefit  from  the  training.  In  its  present 
form,  the  scheme  does  not  allow  quantitative  evaluations  but  trainers  can  rank  different  training 
courses  by  these  characteristics  to  predict  which  training  courses  will  yield  greater  benefits,  including 
better  initial  learning  and  longer  retention. 

Alliger  and  Janak  (1989)  note  several  problems  with  Kirkpatrick’s  classification  scheme,  although 
these  often  arise  from  misunderstandings  of  how  the  scheme  should  be  applied.  First,  trainers  often 
assume  that  the  steps  in  the  scheme  are  arranged  in  ascending  order  of  importance,  which  they  are  not. 
This  may  lead  to  the  view  that  the  best  measures  of  success  of  training  are  based  on  Step  4,  such  as 
reducing  training  costs,  increasing  morale,  and  so  on.  The  relative  importance  of  steps  will  depend  on 
the  goals  of  training.  In  the  case  of  military  training.  Steps  2  and  3  would  be  primary  because  the  need 
is  to  ensure  military  personnel  are  able  to  perform  tasks  when  called  upon. 

A  second  problem  is  that  the  steps  in  the  scheme  do  not  have  clear  causal  links.  Reactions,  for 
example,  do  not  predict  learning;  trainees  may  learn  well  when  training  is  unpleasant  and  involves 
hard  work.  There  should  be  some  causal  links  between  steps  -  learning  and  behavior  should  be 
positively  correlated  -  but  we  do  not  know  the  exact  form  of  these  links. 

In  an  attempt  to  correct  problems  with  Kirkpatrick’s  scheme,  Alliger  et  al.  (1997)  offered  an 
augmented  classification  scheme  based  on  a  meta-analysis  of  relationships  between  training  criteria. 
This  scheme,  shown  in  Table  4.2  below,  expands  the  reactions  and  learning  categories.  At  the 
reactions  level,  Alliger  et  al.  (1997)  divided  trainees  views  of  training  into  affective  and  behavioral  (or 
utility)  components.  Thus,  Step  la  concerns  affective  reactions  such  as  satisfaction,  which  partially 
determines  motivation  to  learn  and  perform.  Step  lb  concerns  the  perceived  utility  of  the  training  for 
subsequent  job  performance,  which  affects  the  degree  to  which  the  training  will  influence  the  trainee 
view  the  training  as  enhancing  ability  to  perform. 


-|Au{tei^d  Training  Criteria  ff 

1.  Reactions 

Reactions 

a.  Affective  reactions 

b.  Utility  judgments 

2.  Learning 

Learning 

a.  Immediate  knowledge 

b.  Knowledge  retention 

c.  Behavior/skill  demonstration 

3.  Behavior 

Behavior 

4.  Results 

Results 

Table  4.2.  Augmented  Training  Criteria  of  Alliger  et  al.  (1997)  Compared  to 
Kirkpatrick’s  Classification  Scheme. 
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At  the  learning  level,  Alliger  et  al.  (1997)  divide  the  category  to  reflect  the  distinction  between 
declarative  and  procedural  knowledge  (e.g.,  Tulving,  1983)  as  well  as  the  initial  level  of  learning. 

Step  2a  is  an  evaluation  of  immediate  post-training  knowledge,  usually  assessed  by  multiple  choice 
tests  or  factual  knowledge.  Level  2b  refers  to  the  long-term  retention  of  declarative  knowledge  needed 
to  perform  the  trained  skill  or  task.  Level  2c  deals  with  the  behavioral  or  skill  retained  from  training, 
which  is  assessed  by  the  actual  performance  of  trained  skills  and  on-the-job  performance.  The 
remaining  two  Steps  are  unchanged  from  Kirkpatrick’ s  original  scheme. 

To  assess  the  validity  of  their  revised  classification  scheme,  Alliger  et  al.  (1997)  conducted  a  meta¬ 
analysis  of  34  articles  examining  training  criteria.  The  meta-analysis  confirmed  earlier  findings 
(Alliger  &  Janak,  1989)  that  inter-correlations  among  levels  are  modest  but  inter-correlations  among 
criteria  within  the  same  level  are  stronger.  Overall,  both  affective  and  utility  reaction  criteria  and 
immediate  and  retained  measures  of  learning  correlated  moderately  with  on-the-job  performance, 
suggesting  some  predictiveness  of  skill  retention.  All  training  criteria  were  reliable  evaluations.  The 
small  number  of  studies  examining  the  effectiveness  of  training  criteria,  however,  limit  their 
usefulness  as  a  model  for  prediction  of  skill  retention  (Alliger  et  al.,  1997). 

4.4.2  Relapse  Prevention  Model 

Marx  (1986)  offers  another  approach  that  is  based  on  individual  characteristics  and  training  outcomes 
as  a  means  of  combating  skill  loss.  The  Relapse  Prevention  Model,  however,  is  more  a  strategy  for 
preventing  skill  loss  than  a  model  for  predicting  the  rate  of  skill  loss.  Nevertheless,  it  can  be  used  as  a 
diagnostic  tool  for  identifying  whether  a  learner  has  the  necessary  support  skills  that  slow  the  decay  of 
learned  skills.  It  can  also  be  used  as  a  training  intervention  to  increase  the  long-term  retention  of 
skills. 


Humansystems  Incorporated 


Skill  Fading  Literature  Review 


Page  35 


Support  skills  are  techniques  and  procedures,  sometimes  explicitly  taught,  that  are  used  to  monitor  and 
enhance  one’s  own  learning  and  performance.  Marx  (1986)  argues  that  any  critical  management  tasks, 
such  as  leadership  and  decision  making,  depend  on  several  support  skills,  such  as  creating  an  effective 
support  network  and  recognizing  behaviors  that  lead  to  errors  or  lapses.  If  an  individual  is  deficient  in 
support  skills,  he  or  she  is  less  likely  to  retain  new  skills  or  knowledge  beyond  the  end  of  training. 
Thus,  Marx  advocates  training  in  support  skills  to  help  trainees  understand  their  value.  With  increased 
awareness  of  where  new  skills  are  vulnerable  to  failure,  managers  can  self-diagnose  support  skill 
deficits. 

The  Relapse  Prevention  Model  helps  trainees  be  more  sensitive  to  difficult  aspects  of  future  training 
situations  and  to  identify  potential  causes  of  training  failure  by  using  information  of  past  failures  as 
feedback.  An  implicit  assumption  underlying  the  model  is  that  skill  failure  or  forgetting  is  not  due  to 
decay  alone  but,  at  least  partially,  to  contextual  forgetting  or  the  mismatch  between  training  and 
performance  environments  (e.g.,  Godden  &  Baddeley,  1975;  Tulving,  1983).  For  example,  a  training 
course  might  teach  a  manager  delegating  skills  but,  when  back  in  the  actual  work  environment,  the 
characteristics  of  the  training  situation  are  largely  unavailable.  As  a  result,  the  manager  does  not  have 
access  to  many  of  the  cues  needed  to  initiate  and  monitor  the  trained  skill  and  so  performs  worse  than 
at  the  end  of  training.  Moreover,  a  working  environment  often  presents  frustrating  obstacles  to 
employing  new  skills,  inducing  the  person  to  revert  to  old  skills.  Consequently,  Marx  (1986)  argues 
that  managers  need  to  be  taught  to  anticipate  the  on-the-job  environment  and  recognize  the  need  for 
support  skills  to  enhance  the  transfer  from  training  to  work  environment.  The  Relapse  Prevention 
Model  identifies  self-control  strategies  to  aid  this  process. 
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Relapse  prevention  strategies 


Strategy 

(S)  Anticipate  and  monitor 
potential  difficulties 

Purpose 

Example 

Proposed  Remedy 

Understand  relapse  process. 

Helps  anticipate  the  RP 
process. 

No  time  to  concentrate  on  skill 
retention. 

Return  to  work  expecting 
difficulty  in  skill  retention. 

Observe  differences  between 
training  and  work  setting. 

Decreases  unpredictability  of 
hectic  work  environment  on 
retention. 

Little  feedback  from  my 
supervisors  and  co-workers. 

Set  time  for  participative 
decision  making  with  my  work 
group  and  supervisor. 

Create  an  effective  support 
network. 

Helps  identify  co-workers  who 
will  support  skill  retention. 

After  training,  it’s  business  as 
usual. 

Meet  with  and  observe  other 
trained  workers. 

Expect  subordinates’ 
skepticism  of  new  behaviors. 

Reduces  perceived  insincerity 
when  implementing  new  skill. 

Co-workers  might  appreciate, 
but  won’t  trust,  my  sudden 
change. 

Announce  participative 
decision  making,  ask  for  input, 
and  start  only  occasional  use. 

Identify  high-risk  situations. 

Helps  pinpoint  situations  where 
relapse  is  likely. 

-  When  work  piles  up 

-  When  I’ve  been  working  too 
hard  and  feel  tense. 

Schedule  first  group  decision 
meeting  during  a  lull  in  the 
week. 

Avoid  implementing  new  skills 
in  overwhelming  situations. 

Allows  choice  of  appropriate, 
safe  setting. 

Usually  use  new  skills  at  the 
first  opportunity  even  if 
conditions  are  unfavorable. 

Schedule  meeting  two  weeks 
after  training,  when  the  backing 
of  work  has  diminished. 

Recognize  seemingly 
unimportant  behaviors  that 
lead  to  errors. 

Strengthens  awareness  of 
behaviors  that  leave  managers 
overwhelmed. 

Inadvertently  scheduling 
participative  decision  last. 

(0)  Increase  rational  thinking 

Reduce  dysfunctional 
emotions. 

Diminishes  irrational  guilt  or  self¬ 
blame  for  early  lapses. 

Self-blame  for  not  mastenng 
group  decision  making  quickly. 

Expect  to  feel  some  self-blame 
but  allow  feeling  to  pass. 

Retain  self-confidence  despite 
temporary  errors. 

Reframes  temporary  lapses  as 
expected. 

Angered  by  subordinates,  drop 
the  whole  idea,  thus  lowering 
my  confidence. 

Perceive  the  lapse  as  a  unique 
event  caused  by  inadequate 
skill  development  not  personal 
inadequacy. 

(B)  Diagnose  and  practice 
related  support  skills 

Diagnose  necessary  support 
skills. 

Enables  assessment  of  support 
skills. 

Poor  time-management  skills 
sabotage  effective  participatory 
decision  making. 

Improve  time  management 
skills  via  a  course. 

Review  disruptive  lifestyle 
patterns. 

Identifies  persistent  personal 
habits  that  can  disrupt  skill 
retention. 

A  perfectionist,  worried  about 
not  doing  things  right,  doesn’t 
try. 

Consider  personal  counseling. 
Perfectionism  may  seem  less 
with  exercise.  Keep  up  health 
and  fitness  programs. 

Mix  required  and  desirable 
activities. 

Helps  manager  balance 
enjoyable  and  tedious  activities. 

Unpleasant  tasks  occur  early  in 
the  week.  Slips  are  likely  when 
fun  tasks  aren't  available. 

Meet  with  the  enjoyable  groups 
early  in  the  week. 

(C)  Provide  appropriate 
consequences  for  behavior 

Assess  organization  support 
for  skill  retention. 

Anticipates  where  self-managed 
rewards  must  supplement 
missing  organizational  support. 

Boss  won’t  notice  or  focus  on 
skill  improvement. 

Instruct  boss  about  the  skill. 

Invite  her  to  a  meeting,  and 
ask  for  feedback  on  progress 
at  performance  review. 

Create  meaningful  rewards 
and  punishments  when  they 
otherwise  don’t  exist. 

Teaches  how  to  identify,  create, 
and  implement  appropnate 
consequences. 

Self-punishment  for  skill 
lapses. 

Reward  yourself  for  even 
model  attempts.  Set  specific 
rewards,  long-  and  short-term. 

Figure  4.3  -  Relapse  Prevention  Strategies  (From  Marx,  1986). 
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Another  key  assumption  of  the  model  is  that  skills  will  be  more  effectively  retained  if  the  degree  of 
unpredictability  of  the  working  environment  is  reduced.  This  can  be  accomplished  in  several  ways. 
Managers  can  be  taught  to  expect  slips  or  failures  and  to  use  these  as  data  for  predicting  and  dealing 
with  future  difficulties.  They  can  be  taught  to  view  failures  as  feedback  on  the  effectiveness  and 
completeness  of  training.  Finally,  Marx  (1986)  developed  a  set  of  relapse  prevention  strategies  (Table 
4.3)  that  serve  as  “cognitive  fire  drills.”  These  help  prevent  errors  or  failures  in  skill  performance 
from  becoming  permanent  loss  of  skill  capability. 

The  relapse  prevention  strategies  are  divided  into  four  categories: 

•  Stimulus  situation:  Pertain  to  work  environment. 

•  Thoughts  and  feelings  (Organism):  Pertain  to  the  self. 

•  Behavior:  Pertain  to  one’s  own  behavior. 

•  Consequences  of  behavior:  Pertain  to  interaction  with  the  organization. 

These  categories  can  be  made  the  focus  of  training  to  enhance  support  skills.  The  first  step  in 
implementing  relapse  prevention  strategies  is  to  choose  or  identify  the  skills  to  be  retained.  The 
learner  then  describes  the  skill  in  measurable  terms  so  that  errors  and  slips  can  be  detected.  After 
identifying  the  skills,  the  trainee  is  presented  with  relapse  prevention  strategies  (see  Table  4.3)  to 
apply  during  training  and  afterwards  in  the  work  environment.  The  final  activity  is  for  trainees  to 
predict  the  circumstances  of  errors  and  lapses  in  performing  skills.  The  trainee  creates  a  detailed 
image  or  mental  simulation  of  the  scenario  so  that  when  it  begins  to  happen  in  the  work  setting,  the 
trainee  can  recognize  it  and  apply  relapse  prevention  strategies. 

Marx  (1986)  argues  that  large  organizations  can  use  the  Relapse  Prevention  Model  as  part  of  a  training 
needs  assessment  process.  By  identifying  the  most  common  support  skill  deficits,  training  objectives 
can  be  chosen  to  optimally  meet  training  needs.  The  model  also  aids  in  developing  training  schedules, 
choosing  training  style,  and  assessing  skill  loss  on-the-job. 

4.4.3  Model  of  Job  Performance  Determinants 

McCloy,  Campbell,  and  Cudeck  (1994)  developed  the  Model  of  Job  Performance  Determinants  as  a 
descriptive  model  of  the  factors  that  directly  determine  individual  differences  in  job  performance. 
Although  this  model  was  not  intended  to  predict  skill  loss  rate,  it  might  be  adaptable  to  this  purpose. 
The  model  is  based  on  the  assumption  that  task  performance  is  multidimensional  and  that  for  any  task 
there  are  a  number  of  performance  components  that  differ  in  their  intercorrelation  with  other  variables. 

The  main  determinants  of  performance  in  this  model  are  all  individual  factors: 

•  Declarative  Knowledge  (DK):  Facts,  rules,  principles,  and  procedures. 

•  Procedural  Knowledge  and  Skill  (PKS):  Automatized  procedures,  routines,  motor 
programs. 

•  Motivation  (M):  Combined  effect  of  three  choice  behaviors;  choice  to  expend  effort, 
choice  of  level  of  effort  to  expend,  and  choice  to  persist  in  expenditure  of  effort. 

Job  Performance  Components  (PCs),  then,  are  a  function  of  DK,  PKS,  and  M.  All  other  factors  are 
hypothesized  in  this  model  to  indirectly  affect  PCs  by  their  effects  on  DK,  PKS,  and/or  M.  Good 
supervisors,  for  example,  can  enhance  DK  and  M,  leading  to  better  performance.  The  effects  of 
situational  and  training  factors,  then,  can  be  predicted  by  knowing  their  effects  on  the  three  underlying 
factors. 

There  is  little  empirical  support  for  McCloy  et  al.’s  (1994)  claim.  In  one  test,  they  analysed  various 
test  types  to  predict  which  factors  would  be  determinants  of  performance.  Performance  on  job 
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knowledge  tests,  they  hypothesized,  would  be  a  function  of  DK,  that  is  job  knowledge.  Performance 
on  work  sample  tests,  which  measure  capability  to  perform  tasks,  would  be  a  function  of  both  DK  and 
PKS.  Supervisor  ratings  of  performance  would  be  a  function  of  all  three  factors,  DK,  PKS,  and  M. 
McCloy  et  al.  (1994)  collected  performance  data  from  first-tour  soldiers  in  19  MOSs,  including: 

•  Hands-on  performance  tests. 

•  Job  knowledge  tests. 

•  Records  of  awards  and  letters. 

•  Task  ratings  by  supervisors  and  peers. 

•  Cognitive  ability  tests. 

•  Spatial  ability  tests. 

•  Perceptual  speed  and  psychomotor  tests. 

In  support  of  their  predictions,  McCloy  et  al.  (1994)  found  that  cognitive  ability  tasks  were  most 
predictive  of  DK  and  moderately  predictive  of  PKS;  there  was  little  relation  with  M.  Psychomotor 
tests  were  most  predictive  of  PKS  and  moderately  predictive  of  DK  but  showed  no  relation  with  M. 
Personality  variables  were  strongly  predictive  of  M  but  showed  little  relation  with  DK  and  PKS. 

4.4.4  Applicability  and  Practicality 

The  training  criteria  approach  was  designed  to  apply  to  any  kind  of  training  and,  hence,  any  kind  of 
skill.  Because  this  model  addresses  the  nature  of  training,  it  is  not  important  to  characterize  the  task  in 
any  great  detail.  The  approach  is  also  easy  and  inexpensive  to  use.  It  can,  in  principle,  be  applied 
once  for  a  given  training  course  to  generate  predictions  of  skill  retention  any  future  time  the  course  is 
administered.  In  practice,  however,  training  criteria  would  have  to  be  evaluated  each  time  a  training 
course  is  conducted  because  a  course  will  vary  from  trainer  to  trainer. 

The  main  disadvantage  of  the  training  criteria  approach  is  that  it  has  not  been  developed  into  a 
quantitative  model.  It  offers  broad  indications  of  how  quickly  skills  will  fade  based  on  the  kind  of 
training  given  but  cannot  indicate  the  rate  of  skill  loss  or  the  asymptote  level  of  performance  in  units 
of  real  time.  This  approach  also  has  received  relatively  weak  empirical  support,  suggesting  that  the 
scheme  as  currently  formulated  is  too  simple  to  provide  accurate  predictions  of  skill  retention  (Alliger 
&  Janak,  1989). 

The  Relapse  Prevention  Model  suffers  from  similar  problems.  Again,  the  model  makes  no 
quantitative  prediction  of  skill  retention  and  it  is  unclear  how  this  capability  could  be  developed  within 
the  framework  laid  out  by  Marx  ( 1986).  The  model  is  aimed  at  providing  strategies  to  maintain  skills 
rather  than  prediction  and  would  probably  be  more  useful  as  a  technique  to  improve  the  effectiveness 
of  refresher  training.  Moreover,  the  model  was  developed  for  managerial  skills,  which  tend  to  be 
highly  cognitive.  Thus,  the  model  may  not  apply  to  primarily  procedural  and  motor  skills  common  in 
the  military.  The  Relapse  Prevention  Model,  however,  would  entail  only  moderate  effort  and  cost  to 
implement,  primarily  in  the  form  of  additional  training  requirements  for  trainers,  commanders,  and 
other  military  personnel. 

The  Model  of  Job  Performance  Determinants  applies  to  all  kinds  of  skills,  given  its  focus  on  the 
learning  and  intrinsic  motivation  of  trainees.  It  is  unclear  what  effort  and  resources  would  be  required 
to  apply  model  in  assessing  DK,  PKS,  and  M.  Given  that  these  are  based  on  individual  factors, 
however,  it  would  presumably  require  fair  amounts  of  manpower  and  resources  to  assess  DK,  PKS, 
and  M  for  each  soldier  and  each  skill. 

This  approach  has  not  been  fully  developed  and  researchers  have  yet  to  describe  the  functions  relating 
DK,  PKS,  and  M  to  performance.  Thus,  this  model  provides  no  quantitative  predictions  of  skill 
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retention,  although  it  might  with  further  development.  There  needs  to  be  systematic  analysis  of 
correlations  between  factors  and  DK,  PKS,  and  M.  Empirical  evidence  so  far  has  been  limited; 
additional  support  is  needed  to  validate  this  approach. 

4.5  The  ARI  Model  of  Skill  Retention 

The  most  fully  developed  model  of  skill  retention  is  the  User’ s  Decision  Aid  (UDA),  developed  by  the 
U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  (Rose,  Radtke,  Shettel,  & 
Hagman,  1985a;  Rose,  Czamolewski,  Gragg,  Austin,  Ford,  Doyle,  &  Hagman,  1985b).  This  model 
was  developed  specifically  to  provide  quantitative  predictions  of  skill  retention  for  military  tasks  and 
so  represents  a  major  advance  from  the  qualitative  models  reviewed  so  far. 

4.5.1  Theoretical  Bases 

Much  of  the  empirical  literature  discussed  in  Section  3. 1  served  as  a  basis  for  the  UDA  model.  As 
researchers  uncovered  the  personal,  task,  training,  and  job  condition  factors  that  affect  skill  retention, 
they  began  to  work  on  descriptive  procedures  to  identify  more  precisely  how  these  factors  produced 
deterioration  of  skills  (e.g.,  Hurlock  &  Montague,  1982).  The  ARI  established  their  modeling  effort  to 
develop  and  validate  a  convenient,  practical  method  for  unit  commanders  and  training  managers  to 
decide  upon  the  allocation  of  training  resources  to  maximize  combat  readiness  (Rose  et  al.,  1985b). 

The  first  issue  dealt  with  in  the  ARI  initiative  was  the  decision  as  to  which  factors  were  relevant  to 
prediction  of  skill  loss  as  well  as  feasible  for  use  in  prediction.  The  ARI  rejected  the  approach  of 
testing  large  numbers  of  soldiers  on  each  Army  task  to  empirically  derive  individual  retention  curves 
because  this  would  require  far  too  many  resources  (Rose  et  al.,  1985b).  Instead,  they  sought  to 
identify  relevant  factors  already  established  at  that  time  in  the  research  literature,  using  these  to 
develop  theoretic  retention  curves  for  use  in  prediction  of  skill  loss. 

Several  ARI  projects  investigated  the  effects  of  training  variables,  including  overlearning  (Hagman  & 
Rose,  1983),  practice  schedules  (Schendel  &  Hagman,  1982),  and  innovative  training  techniques 
(Rose  et  al.,  1985b),  all  of  which  affect  the  rate  of  skill  loss3.  In  addition,  ARI  studies  identified  a 
number  of  task  characteristics  that  have  dramatic  effects,  such  as  task  difficulty,  number  of  steps 
required,  inter-step  cueing,  and  step  relevance  (Rose  et  al.,  1985b).  In  contrast,  studies  revealed  only 
inconsistent  effects  of  individual  ability  measures  such  as  the  AFQT  and  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB)  (Rose  et  al.,  1985b),  although  other  studies  suggest  predictable  effects  of 
personal  factors  (Ackerman  &  Kanfer,  1986;  Ford  et  al.,  1992). 

Deciding  that  training  and  individual  factors  presented  certain  practical  problems  of  assessment,  ARI 
focused  on  task  factors  as  predictors  (these  are  summarized  in  Table  4.4)4.  Wisher  et  al.  (1999)  review 
the  major  factors  identified.  One  of  the  major  predictors  of  skill  loss  is  task  complexity,  which  can  be 
divided  into  three  sub-factors:  number  of  steps  in  a  task,  whether  the  steps  must  be  performed  in  a  set 
sequence,  and  whether  there  is  built-in  feedback  that  indicates  correct  performance  of  steps.  As  the 
number  of  steps  increases,  retention  performance  of  tasks  decreases  (Knerr  et  al.,  1984,  cited  in 
Wisher  et  al.,  1999).  This  effect  is  modified,  however,  by  the  organization  of  steps  in  the  task. 


3  Several  ARI  studies  did,  however,  fail  to  replicate  the  finding  that  retention  is  related  to  the  degree  of  initial 
learning  (Rose  et  al.,  1985b) 

4  There  is  inherently  greater  variability  in  the  way  training  is  administered  for  a  task  from  one  unit  to  another 
than  in  the  way  the  task  is  actually  performed. 
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Memory  for  order  is  often  poor,  which  makes  tasks  within  which  steps  can  be  performed  in  any  order 
easier  to  retain  than  tasks  within  which  steps  must  follow  a  particular  order.5  Nevertheless,  tasks  in 
which  steps  must  be  performed  in  one  specific  order  are  generally  not  that  difficult  to  retain  compared 
to  tasks  in  which  some  steps  must  be  performed  in  order  but  others  can  be  performed  in  any  sequence. 
Single-sequence  tasks  simplify  the  problem  of  remembering  order  and  present  clearer  cues  to  the 
correct  sequence  of  steps.  Tasks  that  naturally  present  feedback  concerning  the  accuracy  of  steps  can 
further  aid  retention  of  a  skill  by  reducing  errors  and  preventing  an  individual  from  proceeding  down 
the  wrong  track  before  realizing  an  error. 

Another  set  of  task  factors  pertain  to  the  demands  the  task  places  on  performers  (Wisher  et  al.,  1999). 
One  such  factor  is  the  amount  of  information  or  number  of  facts  that  the  performer  must  recall  in  order 
to  perform  the  task.  Consistent  with  the  “magic  number  7±2”  (Miller,  1956),  people  can  easily 
remember  5  to  9  chunks  of  information  but  encounter  difficulty  with  more  than  9  chunks,  unless  they 
employ  some  strategy  to  subjectively  re-code  the  information  into  fewer  than  9  chunks  (e.g.,  Simon, 
1974).6  The  availability  of  such  strategies  will  itself  depend  on  the  nature  of  the  information;  abstract 
information  may  be  difficult  to  re-organize. 

Tasks  can  also  place  execution  demands  on  performers.  As  noted  earlier,  continuous  tasks  tend  to  be 
retained  better  than  discrete  tasks  (Arthur  et  al.,  1998;  Hurlock  &  Montague,  1982),  presumably 
because  they  require  an  intermediate  degree  of  motor  control  and  involve  less  sequencing  of  steps. 
Memory  for  cognitive  skills  also  tends  to  be  stable  for  long  periods  of  time,  although  they  do  exhibit 
forgetting  (Wisher  et  al.,  1999).  People  tend  to  show  greater  skill  loss  for  cognitive  skills  than  for 
perceptual-motor  skills. 

To  combat  the  limitations  of  human  cognition,  the  designers  of  tasks  often  create  job  and  memory  aids 
to  help  performers  remember  steps  as  well  as  necessary  information.  These  vary  in  quality  and 
comprehensiveness  but  the  availability  of  an  effective  job  aid  helps  individuals  retain  relatively  high 
levels  of  performance  over  time. 

Time  limits  inherent  to  a  task  can  impair  retention  of  skills.  An  aspect  of  skill  loss  is  the  slowing  of 
performance  as  performers  take  more  time  to  remember  and  execute  steps.  As  a  result,  a  time  limit 
can  impair  performance  by  preventing  the  performer  from  finishing  or  by  creating  stress  that  hinders 
the  performer’s  expression  of  skill. 
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1. 

Number  of  steps 

2. 

Whether  steps  must  be  performed  in  a  set  sequence 

3. 

Whether  the  task  contains  feedback  that  indicates  the  correct  order  of  steps 

D 

Number  of  facts  or  information  chunks  that  must  be  recalled 

£ 

Execution  demands 

5  This  effect  is  likely  confounded  with  the  effect  of  overall  number  of  steps  because  tasks  in  which  steps  can  be 
performed  in  any  order  presumably  tend  to  have  fewer  steps  than  sequential  tasks. 

6  A  chunk  is  typically  defined  as  an  elementary  unit  of  meaningful  information,  so  that  a  chunk  depends  on  how 
an  individual  organizes  information. 
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6. 

Whether  the  skill  is  cognitive  or  perceptual/motor 

7. 

Whether  there  are  job  and/or  memory  aids  for  the  task 

8. 

The  time  limit  of  the  task  (if  any) 

Table  4.4.  -  Task  Factors  Identified  During  the  Development  of  the  UDA  Model  of  Skill  Retention 


4.5.2  Development  of  the  Model 

The  major  responsibility  of  the  ARI  project  was  to  determine  a  way  to  keep  soldiers  proficient  in  tasks 
(Rose  et  al.,  1985a).  Given  that  soldiers  will  forget,  the  ARI  concentrated  on  developing  a  tool  to 
increase  the  efficiency  of  scheduling  refresher  training.  Training  time  should  be  spent  on  tasks  in 
which  soldiers  have  dropped,  or  are  about  to  drop,  below  a  criterion  level  of  performance.  To 
schedule  training  when  needed  (but  not  before),  the  ARI  developed  the  UDA  model  on  the  basis  of  the 
task  factors  identified.  The  model  predicts  the  decay  rate  of  tasks  over  periods  during  which  soldiers 
have  engaged  in  no  practice.  There  is  no  means  by  which  to  account  for  the  effect  of  practice  on  skill 
retention.7 

The  UDA  was  developed  according  to  a  few  basic  steps  (Rose  et  al.,  1985b): 

1.  Identify  task  dimensions  most  likely  to  be  related  to  retention. 

2.  Convert  task  dimensions  into  rating  scales,  develop  anchor  points,  and  analytically 
assign  weights  to  each  point  on  the  scales. 

3.  Assess  each  scale’s  reliability  and  validity  by  having  several  judges  rate  tasks 
on  each  scale. 

4.  Examine  inter-rater  agreement  and  correlation  between  task  ratings  and  actual 
retention  data. 

5.  Iterate  these  steps  to  develop  a  set  of  valid  and  reliable  scales. 

Rose  et  al.  (1985b)  describe  each  of  these  steps  in  detail,  effectively  laying  out  a  “cookbook”  for 
development  of  a  model  of  skill  retention.  The  remainder  of  this  section  will  briefly  describe  the 
methodology. 

The  basis  of  the  UDA  model  is  the  algorithm  that  weights  and  summarizes  the  relevant  task  factors  to 
produce  a  single  task  retention  score.  This  score  is  then  used  to  predict  soldiers’  level  of  task 
proficiency  over  time.  ARI  researchers  developed  the  initial  algorithm  from  the  empirical  literature 
available.  To  refine  the  algorithm,  researchers  then  performed  regression  analyses  on  data  obtained 
from  previous  ARI  studies.  The  regression  equations  obtained  for  the  set  of  eight  task  variables 
(Table  3.3)  indicated  substantial  correlations  between  retention  at  two  months  and  each  factor. 
Correlations  between  retention  at  four  months  and  the  factors  were  not  as  high  but  still  significant. 
Thus,  the  initial  algorithm  suggested  a  viable  means  to  predict  skill  retention. 

To  refine  the  algorithm  further,  ARI  researchers  collected  performance  data  from  soldiers  for  a  sample 
of  13B10  (cannon  crewman)  MOS  tasks.  These  data  was  used  to  empirically  establish  acquisition  and 
retention  functions  (performance  plotted  as  a  function  of  time)  and  to  examine  the  relationships  of  task 
factors  to  retention.  The  MOS  tasks  used  in  this  phase  of  the  project  were  all  basic  skills  pertaining  to 
restricted  duty  positions.  A  total  of  1 1  tasks  could  be  tested  within  the  time  and  equipment  constraints 
of  the  study. 


7  The  UDA  model  assumes  that  initial  training  has  brought  soldiers  to  a  criterion  level  of  one  perfect 
performance  (100%  proficiency)  of  the  task/skill  (Wisher  et  al.,  1999). 
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Based  on  these  early  studies,  researchers  were  able  to  develop  and  implement  the  UDA  procedure  to 
empirically  determine  the  values  for  each  rating  option  of  the  UDA  questions.  The  researchers  also 
improved  the  comprehensibility  of  the  rating  procedure  based  on  user  feedback.  It  is  worth  noting  that 
the  ARI  researchers  created  the  preliminary  model  using  arbitrary  assumptions  rather  than  deriving 
scale  values  empirically.  Furthermore,  the  relative  weights  for  rating  scales  and  the  rule  for  combining 
them  were  also  arbitrary.  The  researchers  did  this  because  they  had  not  had  access  to  a  sufficient  pool 
of  tasks  to  conduct  regression  analyses  necessary  to  determine  best-fit  values,  necessitating  the  initial 
arbitrary  estimates. 

To  increase  the  number  of  cases,  researchers  combined  data  sets  from  multiple  studies  to  create  a  data 
set  containing  data  on  a  total  of  54  tasks.  To  decrease  the  number  of  task  characteristic  predictors, 
they  performed  a  factor  analysis  of  the  task  dimensions  comprising  the  UDA,  which  reduced  the 
number  of  independent  predictors  to  five.  Eight  raters  then  made  ratings  for  the  54  tasks.  From  the 
data  set,  the  researchers  determined  the  percent  soldiers  GO  (i.e.,  the  percentage  of  soldiers  in  the 
study  who  were  at  or  above  the  criterion  performance  level)  and  the  percent  task  steps  GO  (the 
percentage  of  task  steps  for  which  soldiers  were  at  or  above  criterion  performance)8.  Analyses 
resulted  in  empirically-based  scale  values  for  the  answer  options  within  each  UDA  rating  question. 
This  scaling  procedure  is  critical  to  convert  the  retention  level  predicted  in  the  model  into  a  value  for 
“real-world”  retention.  It  also  provided  regression  weights  for  each  UDA  question  to  account  for 
differences  in  the  importance  of  different  factors. 

We  can  break  down  this  procedure  into  five  more  specific  steps.  The  first  step  involved  creating  a 
matrix  of  the  raters’  modal  responses  arrayed  by  UDA  questions  and  the  54  tasks.  In  step  2,  the 
researchers  input  the  mode  matrix  into  an  analysis  of  variance  (ANOVA)  and  conducted  separate 
ANOVAs  for  each  of  the  eight  UDA  questions,  with  percent  soldiers  GO  as  the  dependent  variable 
and  answer  options  for  each  question  as  the  independent  variables.  They  then  generated  cell  means  for 
each  answer  option  of  five  of  the  questions;  the  remaining  three  questions  had  non-significant  F  values 
and  so  weights  were  assigned  arbitrarily.  The  researchers  then  transformed  the  mode  matrix  by 
replacing  each  entry  with  the  corresponding  cell  mean  from  the  ANOVA.  Thus,  the  matrix  values 
corresponded  to  the  mean  percentage  of  soldiers  GO  for  all  tasks  with  a  given  question  option.  For 
example,  this  would  change  a  rating  of  “2”  in  the  question  pertaining  to  the  presence  of  job  aids  to 
“61.3,”  which  was  the  mean  percent  of  soldiers  GO  for  all  tasks  with  “Good”  job  aids.  The  factor 
analysis  indicated  that  four  questions  loaded  significantly  onto  one  factor,  which  could  be  interpreted 
as  the  cognitive  demands  of  the  task.  Thus,  researchers  combined  these  questions  into  a  single 
independent  variable  by  adding  values. 

The  third  step  was  to  multiply  matrix  values  by  the  question  weights.  The  regression  analysis 
examined  the  relationship  between  five  UDA  variables  and  mean  two-month  retention  performance 
across  the  54  tasks.  The  results  indicated  that  79%  of  the  variance  in  performance  was  accounted  for 
by  the  five  predictor  variables. 

Step  4  involved  transforming  the  scale  values.  The  weights  were  rounded  off  to  the  nearest  whole 
number  and  a  constant  added  or  subtracted  for  each  UDA  question  so  that  the  lowest  value  equaled 
zero.  The  sum  of  the  additions  and  subtractions  were  added  to  the  regression  equation  constant  so  that 
the  transformations  had  no  effect  on  the  regressions  but  improved  the  usability  of  the  scales.  The  final 
step  was  to  add  up  the  eight  transformed  scores  for  each  task  to  arrive  at  a  total  UDA  score  for  each 
task. 


8  The  term  GO  indicates  that  personnel  are  capable  of  performing  a  task  successfully. 
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The  purpose  of  the  UDA  model  is  to  generate  numerical  predictions  of  skill  retention.  Thus,  an 
important  issue  is  how  to  relate  predictions  of  the  model  to  actual  performance;  in  other  words,  what 
does  a  UDA  value  of,  say,  86  mean  and  how  can  it  be  used  to  estimate  performance  (Rose  et  al., 
1985b)?  The  translation  of  UDA  value  for  predicting  two-month  retention  performance  is 
straightforward  because  the  option  weights  and  combination  rule  were  developed  from  data  on  two- 
month  retention  performance;  a  UDA  value  of  86  is  a  prediction  of  86%  of  40  (the  constant  in  the 
regression  equation)  or  26%  proficiency  after  two  months.  ARI  researchers  followed  an  iterative 
process  to  develop  a  generative  function  to  allow  prediction  at  any  retention  interval.  They 
hypothesized  a  generation  function,  generated  it  for  two-  and  four-month  retention  intervals,  compared 
the  predicted  and  actual  retention  levels,  then  revised  their  hypothesis  and  repeated  the  procedure. 
Various  functions  were  compared  in  terms  of  the  obtained  correlation  and  absolute  arithmetic  error 
between  actual  and  predicted  retention  scores.  The  best  fitting  function  was: 

Predicted  Proficiency  for  Week  Y  =  100  x  ((UDA/100)1/8)y 

Thus,  retention  performance  after  Y  weeks  of  no  practice  is  predicted  by  dividing  the  UDA  value 
(after  subtracting  the  constant)  by  100,  taking  the  eighth  root,  raising  the  result  to  the  Yth  power,  then 
multiplying  by  100.  For  two  months,  Y  =  8,  and  the  formula  reduces  to  just  the  UDA  score.  When 
used  to  generate  predictions  for  the  four-month  as  well  as  two-month  retention  intervals,  the  ARI  team 
found  a  correlation  of  0.83  between  actual  and  predicted  retention  performance.  The  absolute  error  of 
prediction  was  less  than  one  half  of  one  percent. 

Having  achieved  a  high  degree  of  accuracy  in  prediction,  researchers  turned  to  improving  the  usability 
of  the  UDA.  Actual  and  potential  users  of  the  model  were  interviewed.  Based  on  this  feedback, 
researchers  revised  the  questions  and  instructions  to  improve  comprehensibility  and  convenience  of 
the  UDA  as  a  tool  for  predicting  skill  retention. 

4.5.3  The  UDA  Decision  Aid 

The  UDA  model  was  developed  to  form  the  basis  of  a  decision  aid  tool  to  be  used  in  predicting  skill 
retention  and  the  need  for  refresher  training  in  military  tasks  (Rose  et  al.,  1985a).  Specifically,  the 
UDA  is  used  to  determine  (Rose  et  al.,  1985a;  Wisher  et  al.,  1999): 

•  How  quickly  task  skills  will  be  forgotten. 

•  Which  task  among  several  will  be  forgotten  or  remembered  after  a  specified  interval. 

•  What  percentage  of  soldiers  will  be  able  to  perform  a  task  after  up  to  one  year  without 
practice. 

•  When  to  conduct  refresher  training  to  keep  a  group  at  a  criterion  level. 

The  UDA  employs  ratings  of  task  characteristics  to  predict  skill  retention.  The  ratings  are  made  by 
individuals  with  extensive  knowledge  of  the  task.  Optimally,  more  than  one  person  performs  task 
ratings  to  ensure  accuracy  (Rose  et  al.,  1985a).  Differences  in  ratings  must  be  resolved  by  discussion 
because  the  UDA  accepts  only  certain  values  and  averages  of  ratings  have  no  meaning  in  the  model. 
All  that  is  required  to  make  task  ratings  is  a  task  summary  that  provides  descriptive  information  that 
can  help  resolves  differences  of  judgment  and  reduce  the  subjectivity  of  ratings  (Rose  et  al.,  1985a).  If 
no  task  summary  is  available,  having  raters  write  their  own  summary  can  help  them  be  more  thorough 
and  objective. 


State ' 
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1 .  Are  job  or  memory  aids  used  by  the  soldier  in 
performing  (and  in  the  performance  evaluation  of) 

1  (yes),  0  (no) 

Task  Characteristic;  Presence 
of  job  aid 
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this  tasks? 

2.  How  would  you  rate  the  quality  of  the  job  or 
memory  aid? 

56  (excellent),  25  (very 
good),  2  (marginally  good),  1 
(poor) 

Task  Characteristic;  Presence 
of  job  aid 

3.  Into  how  many  steps  has  the  task  been  divided? 

25  (one),  14  (two  to  five),  12 
(six  to  ten),  0  (more  than  ten) 

Task  Characteristic;  Number 
of  steps 

4.  Are  the  steps  in  the  task  required  to  be  performed 
in  a  definite  sequence? 

10  (None),  5  (all),  0  (some 
are  and  some  are  not) 

Task  Characteristic; 
Organization 

5.  Does  the  task  provide  built-in  feedback  so  that 
you  can  tell  if  you  are  doing  each  step  correctly? 

22  (for  all  steps),  19  (for 
greater  than  50%  of  steps), 

1 1  (for  up  to  50%  of  steps),  0 
(for  none) 

Task  Characteristic; 
availability  of  feedback 

6.  Does  the  task  or  part  of  the  task  have  a  time  limit 
for  its  completion? 

40  (no  time  limit),  35  (easy 
time  limit),  0  (difficult  time 
limit) 

Task  Characteristic;  Stress 

7.  How  difficult  are  the  mental  processing 
requirements  of  this  task? 

37  (almost  no  requirements), 
28  (simple  requirements),  3 
(complex  requirements),  0 
(very  complex  requirements) 

Task  Characteristic;  Difficulty 

8.  How  many  facts,  terms,  names,  rules,  or  ideas 
must  a  soldier  memorize  in  order  to  do  the  task? 

20  (none),  18  (one  to  three), 

1 3  (four  to  eight),  0  (more 
than  eight) 

Task  Characteristic;  Difficulty 

9.  How  hard  are  the  facts,  terms,  that  must  be 
remembered? 

1 

34  (not  applicable;  none  to 
remember),  31  (not  hard  at 
all),  12  (somewhat  hard),  0 
(very  hard) 

Task  Characteristic;  Difficulty 

1 0.  What  are  the  motor  control  demands  of  the  task? 

2  (none),  0  (small),  16 
(considerable),  3  (very  large) 

Task  Characteristic;  Difficulty 

Table  4.5.  -  UDA  Questions  and  Scale  Values 


The  UDA  contains  ten  questions  that  raters  answer  based  on  the  task  summary  and  their  knowledge  of 
the  task.  Each  question  has  a  set  of  answers  to  choose  from.  Raters  select  the  appropriate  answer  and 
note  the  scale  value  associated  with  the  selected  answer.  The  questions  and  their  scale  values  are 
listed  in  Table  4.5.  When  all  ten  questions  have  been  answered,  the  raters  compute  the  total  of  the 
scale  values,  which  constitutes  the  task’s  retention  score.  Multiple  raters  review  the  ratings  given  to 
the  task  and  resolve  differences  to  produce  a  final,  agreed-upon  task  retention  score. 

The  UDA  contains  two  Performance  Prediction  Tables  that  are  used  to  convert  the  total  retention  score 
into  a  prediction  of  performance  for  the  rated  task.  The  numbers  in  the  tables  represent  the  expected 
proportion  of  soldiers  in  a  unit  that  are  able  to  perform  the  task  correctly  at  various  retention  intervals 
up  to  one  year  without  practice.  The  UDA  does  not  predict  retention  performance  for  individuals. 

The  first  table  plots  the  expected  proportion  of  soldiers  versus  monthly  intervals,  whereas  the  second 
table  plots  the  expected  proportion  of  soldiers  versus  weekly  intervals  (see  Table  4.6). 
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Predicted  Proportion  of  Soldiers  at  Criterion  Performance  Level 


Months  Since  Last  Performance 

Total 

Score 

1 

2 

3 

4 

5 

6 

7 

8 

180+ 

100 

100 

100 

100 

100 

100 

100 

100 

175 

97 

95 

92 

90 

87 

85 

83 

81 

170 

94 

90 

85 

81 

76 

72 

69 

65 

165 

92 

85 

78 

72 

66 

61 

56 

52 

160 

89 

80 

71 

64 

57 

51 

45 

40 

155 

86 

75 

64 

56 

48 

42 

36 

31 

150 

83 

70 

58 

49 

40 

34 

28 

24 

145 

80 

65 

52 

42 

34 

27 

22 

17 

140 

77 

60 

46 

36 

27 

21 

16 

12 

135 

74 

55 

40 

30 

22 

16 

12 

9 

130 

70 

50 

35 

25 

17 

12 

8 

6 

125 

67 

45 

30 

20 

13 

9 

6 

4 

120 

63 

40 

25 

16 

10 

6 

4 

115 

59 

35 

20 

12 

7 

4 

110 

54 

29 

16 

8 

4 

105 

50 

25 

12 

6 

100 

44 

20 

8 

95 

38 

15 

90 

31 

85 

Table  4.6  •  UDA  Performance  Prediction  Table  (Month  Intervals) 

(Adapted  from  Rose  et  al,  1985a) 

To  find  the  percentage  of  soldiers  expected  to  be  able  to  perform  the  rated  task,  raters  find  the  table 
entry  corresponding  to  the  total  retention  score  and  the  retention  interval  of  interest.  To  determine 
how  frequently  refresher  training  is  needed,  raters  find  the  row  corresponding  to  the  total  retention 
score  then  locate  the  table  entry  of  50%  (or  any  other  criterion).  The  retention  interval  at  which  the 
criterion  occurs  is  the  maximum  interval  for  refresher  training  to  sustain  a  unit  at  the  criterion  level. 
The  tables  can  also  be  used  to  determine  what  level  of  proficiency  is  expected  for  a  unit  that  receives 
refresher  training  at  a  given  interval  of  time  (e.g.,  every  X  months).  The  expected  proficiency  is  given 
by  the  table  entry  corresponding  to  the  total  retention  score  and  the  interval  (X  months). 

4.5.4  Empirical  Evaluation 

The  ultimate  goal  of  any  model  of  skill  retention  is  to  accurately  predict  performance  as  a  function  of 
time.  This  is  tested  by  comparing  predictions  of  retention  performance  made  by  a  model  to  observed 
retention  performance  of  individuals  under  representative  conditions.  The  UDA  model  has  been 
extensively  examined  and  evaluated  by  ARI  researchers.  However,  many  studies  cited  in  support  of 
the  UDA  (such  as  Macpherson,  Patterson,  &  Mirabella.,  1989;  Sabol,  Chapell,  &  Meiers,  1990; 
Wisher  et  al.,  1991;  Wisher,  Sabol,  &  Ozkaptan,  1996;  see  Wisher,  Sabol,  &  Ellis,  1999)  did  not 
actually  compare  predicted  retention  performance  to  actual  retention  performance.  As  such,  they  do 
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not  provide  a  true  validation  of  the  UDA  because  it  cannot  be  determined  how  accurate  is  the  UDA  in 
predicting  skill  retention. 

In  fact,  only  one  study  reports  a  comparison  between  UDA  predictions  and  actual  retention  data.  Rose 
et  al.  (1985b)  used  data  collected  from  a  sample  of  22  13B10  (cannon  crewman)  MOS  tasks  that  had 
not  been  used  in  the  development  of  the  UDA  to  compare  predictions  of  the  UDA  to  actual  skill  loss. 
They  calculated  Pearson  product-moment  correlations  between  actual  and  predicted  scores  for  percent 
soldiers  GO  and  percent  steps  GO.  For  all  tasks,  these  correlations  were  significant,  although 
predicted  means  were  about  four  points  lower  than  actual  scores  at  the  two-month  retention  interval. 
The  tendency  to  underestimate  scores  increased  with  the  length  of  the  retention  interval.  Rose  et  al. 
(1985a)  suggest  that  numerous  factors,  such  as  experimental  manipulations  of  feedback  and  baseline 
proficiency,  events  during  the  retention  interval,  and  individual  differences  may  account  for  the 
underestimation  of  proficiency. 

4.5.5  Applicability  and  Practicality 

The  UDA  is  one  of  the  few  approaches  for  which  empirical  research  has  been  done  to  assess  its 
applicability  and  practicality.  In  one  study,  Macpherson,  Patterson,  and  Mirabella  (1989)  used  the 
UDA  to  predict  skill  retention  for  a  set  of  nine  63W10  MOS  (vehicle  maintenance)  tasks  (see  Table 
4.7).  One  of  the  goals  was  to  determine  whether  the  model  could  be  used  easily  at  the  Aberdeen 
Proving  Grounds.9  Eight  instructors  from  the  Ordinance  School’s  Wheeled  Vehicle  Department  rated 
each  task  using  the  UDA  and  task  lists  prepared  from  Army  technical  manuals.  The  task  lists  included 
task  steps,  indicated  which  steps  had  to  be  performed  in  order,  end-products  of  subtasks,  safety 
requirements,  and  tools  and  materials  needed. 


9  U.S.  Army  facility,  Maryland. 
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Diagnose  hard  starting 
Diagnose  loss  of  engine  power 
Diagnose  stalling  engine 

Diagnose  engine  which  cranks  but  does  not  start 
Replace  fuel  injector  pump 
Remove  and  replace  steering  gear  on  an  M35A2 
Remove  and  replace  axle  on  an  M813A1 
Remove  and  replace  clutch  on  an  M35A2 
Remove  and  replace  transmission  on  an  M35A2 


□L— 


Table  4.7.  -  Vehicle  Maintenance  Tasks  Examined  by  Macpherson  et  al.  (1989). 


The  eight  instructors  took  a  total  of  26  hrs,  or  2.8  hrs  per  task,  to  complete  their  ratings  and  determine 
predicted  retention.  The  total  clock  time  involved  was  10  hrs,  or  1.0  hr  per  task.  Each  rater,  working 
independently  with  an  interviewer,  took  one  hour  to  rate  five  tasks,  or  12  minutes  per  task,  per  rater.  A 
group  discussion,  led  by  an  interviewer,  took  one  hour  for  each  of  two  groups  of  raters. 

Macpherson  et  al.  (1989)  were  able  to  generate  retention  curves  for  the  nine  tasks.  These  curves 
served  as  theoretical  estimates  of  pure  forgetting  of  unit  task  skills  without  intervening  practice.  From 
these  results,  Macpherson  et  al.  (1989)  were  able  to  predict  that  tasks  receiving  total  scores  above  180 
would  not  be  forgotten  within  one  year,  the  maximum  interval  spanned  by  the  UDA.  Skills  for  low  to 
moderate  scoring  tasks  should  decay  very  rapidly  compared  to  skills  for  tasks  with  higher  scores. 
Based  on  the  results,  Macpherson  et  al.  (1989)  predicted  the  rank  order  of  scales  that  differentiated 
among  tasks  in  decreasing  effect  on  retention: 

•  Technical  manual  quality. 

•  Mental  requirements. 

•  Feedback  provided  by  task. 

•  Number  of  facts  to  memorize. 

•  Ease  of  recalling  memorized  facts. 

•  Physical  skill  required. 

This  study  demonstrated  that  it  was  feasible  to  use  the  UDA  to  differentiate  and  prioritize  critical 
tasks.  The  raters  were  able  to  identify  those  tasks  most  susceptible  to  skill  decay  and  make  predictions 
of  when  refresher  training  would  be  needed.  The  study  also  demonstrated  the  practicality  of  the  UDA. 
The  use  of  the  UDA  is  a  cost-effective  supplement  to  field  studies  that  requires  few  materials  and 
relatively  little  effort.  The  raters  had  no  difficulty  reaching  agreement  on  scale  values  for  each 
question  of  the  UDA  for  each  task.  Thus,  the  resolution  procedure  of  the  UDA  seems  to  work  well 
and  allows  multiple  raters  to  achieve  a  single  retention  score. 

In  a  larger  scale  study,  Sabol,  Chapell,  and  Meiers  (1990)  used  the  UDA  model  to  make  predictions  of 
the  decay  functions  for  85  skills  needed  to  operate  Mobile  Subscriber  Equipment  (MSE)  (a 
communication  system).  MSE  is  a  radio  telephone  system  and  its  operation  entails  many  complex, 
highly  cognitive  skills  within  MOS  3  ID  (MSE  Network  Transmission  Operator)  and  MOS  3 IF  (MSE 
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Network  Switch  Operator).  The  purpose  of  their  study  was  to  predict  retention  of  MSE  operator  skills 
and  to  expand  the  UDA  to  cover  more  complex  skills  than  those  examined  in  its  development. 

Sabol  et  al.  (1990)  had  six  SMEs  from  Delta  Company,  442nd  SIG  BG  serve  as  raters.  Three  of  the 
SMEs  were  expert  on  MOS  3 ID  procedures  and  three  on  MOS  3 IF  procedures.  Using  exhaustive 
task  inventories  generated  during  the  development  of  the  MSE’s  Soldier’s  Manuals  and  lists  of  skills 
generated  during  field  evaluation  of  MSE,  the  raters  completed  the  ten  question  rating  procedure  of  the 
UDA  for  all  85  tasks.  Each  SME  was  interviewed  separately  and  took  about  two  hours  for  MOS  3  ID 
tasks  and  about  three  hours  for  MOS  3  IF  tasks.  Sabol  et  al.  (1990)  reported  that  they  intended  to 
obtain  data  with  which  to  compare  predictions  of  the  percentage  of  soldiers  GO  for  each  task  to  actual 
performance  data  of  MSE  operators.  However,  we  were  unable  to  identify  any  publication  reporting 
the  results  of  such  a  comparison. 

Predictions  of  retention  have  been  similarly  computed  for  “peace  support  operations”  tasks  (Wisher  et 
al.,  1996).  As  in  the  Sabol  et  al.  (1990)  study,  SMEs  rated  a  number  of  tasks  and  these  ratings  served 
as  a  basis  for  computing  skill  retention  curves  using  the  UDA.  Although  no  comparison  to  actual 
retention  performance  was  made,  the  UDA  predictions  were  sufficiently  precise  to  allow  researchers 
to  rank  order  tasks  in  terms  of  their  expected  difficulty  and  susceptibility  to  forgetting.  This  rank 
order  was  distributed  in  a  guide  job  aid  to  Army  trainers  for  Operation  Joint  Endeavor  in  Bosnia. 

The  UDA  has  been  demonstrated  to  be  relatively  easy  to  administer.  Some  training  is  needed  to 
perform  the  ratings  correctly  and  reliably  but  this  training  requirement  does  not  seem  to  be  excessively 
burdensome  (Radtke  &  Shettel,  1985).  The  UDA  itself  is  low-cost  and  requires  no  special  equipment. 

The  applicability  of  the  UDA  for  predicting  retention  of  procedural  tasks  seems  well  established.  It  is 
less  clear  how  well  this  model  applies  to  predominantly  cognitive  or  continuous  motor  tasks.  The 
questions  that  make  up  the  UDA  may  be  adapted  to  these  kinds  of  tasks  but  they  seem  less  central. 
Moreover,  the  UDA  was  developed  in  the  context  of  a  variety  of  specialized  skills  drawn  from  U.S. 
MOS  categories.  Additional  validation  with  basic  and  command  skills  is  needed  to  assess  the  model’s 
applicability. 

Another  potential  problem  with  the  UDA  model  is  that  it  may  neglect  important  factors.  By  relying 
exclusively  on  task  factors,  the  UDA  achieves  a  high  degree  of  ease-of-use  but  may  sacrifice  accuracy 
of  prediction.  Factors  such  as  the  degree  of  overlearning  (Hagman  &  Rose,  1983;  Lance  et  al.,  1998; 
Schendel  &  Hagman,  1980,  cited  in  Hagman  &  Rose,  1983),  opportunity  to  practice  (Blankmeyer, 
1998;  Ford  et  al.,  1992;  Landry  &  Campbell,  1997),  similarity  of  training  and  performance 
environments  (Healy  et  al.,  1992;  Marmie  &  Healy,  1995),  and  possibly  individual  aptitude  (Earles  & 
Ree,  1992;  Hurlock  &  Montague,  1982)  have  strong  effects  on  skill  retention.  These  factors,  however, 
are  more  difficult  to  assess  than  task  factors  because  they  vary  from  person  to  person  or  training 
session  to  training  session.  Nevertheless,  in  light  of  the  questions  concerning  the  true  validity  and 
applicability  of  the  UDA,  it  seems  worth  considering  the  role  of  training,  individual,  and  retention 
interval  factors  in  predicting  skill  retention. 

4.6  Other  Quantitative  Approaches 

The  UDA  is  the  most  comprehensive  and  developed  approach  to  making  quantitative  predictions  of 
skill  retention.  Nevertheless,  researchers  have  suggested  other  approaches,  which  will  be  discussed  in 
this  section. 
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4.6.1  Correlational  Approaches 

A  fundamental  tenet  of  research  examining  the  effects  of  various  factors  on  skill  retention  has  been 
that  accurate  predictions  of  skill  loss  can  be  made  when  the  relationship  between  those  factors  and 
skill  retention  is  made  clear.  That  was  the  approach  taken  by  the  developers  of  the  UDA,  who 
regressed  a  number  of  factors  on  skilled  performance  to  determine  their  relation  to  retention  (Rose  et 
al.,  1985a).  Other  researchers  have  performed  similar,  but  much  simpler,  regression  analyses  to 
correlate  the  effects  of  one  or  two  factors  with  retention. 

One  approach  is  to  derive  the  empirical  relationship  between  retention  performance  and  the  retention 
interval.  Researchers  examine  retention  of  a  skill  at  a  few  specific  intervals  then  determine  the  best 
fitting  function  that  describes  the  relation  of  retention  to  time  (e.g.,  Grant  &  Logan,  1993;  Wixted  & 
Ebbensen,  1991).  This  relationship  is  expressed  as  a  power  function  that  can  be  used  to  generate 
predictions  at  any  retention  interval.  The  predictiveness  of  empirically  derived  functions  depends  on 
several  factors: 

•  The  representativeness  of  the  tested  sample  to  the  population  of  prediction. 

•  The  representativeness  of  the  tested  skills  to  the  skills  for  which  predictions  are  made. 

•  The  reliability  of  the  retention  data  used  to  generate  the  power  function. 

To  afford  prediction,  a  sample  should  be  drawn  from  the  population  for  which  prediction  will  be  made. 
What  constitutes  an  appropriate  sample  depends  on  the  tasks  under  consideration.  Basic  psychomotor 
processes  differ  very  little  across  people  from  diverse  backgrounds.  Consequently,  accurate  prediction 
can  often  be  made  from  samples  of  college  students  to  virtually  any  population.  Tasks  involving  more 
cognitive  skills,  however,  are  affected  a  great  deal  by  the  particular  educational,  social,  and  work 
environments  of  the  people  performing  the  tasks.  As  a  result,  samples  that  differ  even  slightly  from  the 
population  under  consideration  can  lead  to  functions  that  are  poorly  predictive. 

Another  approach  is  to  determine  the  relationship  of  specific  task,  learner,  training,  and  retention 
interval  factors  to  retention.  Given  the  large  number  of  factors  that  have  been  found  to  have 
measurable  effects  on  skill  retention,  however,  it  is  difficult  to  envision  a  single  correlational 
relationship  being  a  very  informative  predictor.  That  is  why  researchers  have  focused  on  a  few  factors 
that  seem  to  be  especially  critical  and  very  convenient  to  assess.  One  of  these  is  the  degree  of  initial 
learning.  First  of  all,  this  factor  encompasses  a  number  of  training  and  individual  factors  that 
determine  how  much  trainees  learn.  Secondly,  most  training  techniques  involve  an  end-of-course 
assessment  of  some  sort,  so  that  obtaining  a  measure  of  degree  of  learning  is  convenient  and  may  not 
create  additional  costs  or  burdens  for  trainers. 

In  one  study,  Goldberg  and  O’Rourke  (1989)  examined  the  effects  of  amount  of  initial  training, 
training  workload,  and  length  of  retention  interval  on  performance  in  a  manual  tracking  task  (skiing). 
Participants  trained  to  error-free  performance  then  returned  after  a  one  to  six  week  period  of  no¬ 
practice  for  retraining  and  assessment.  The  training  workload  and  retention  interval  were  varied 
between  participants  and  amount  of  training  within-participants.  The  measures  of  retention  were  run 
completion  times  and  number  of  retraining  runs  to  re-achieve  criterion.  The  results  showed  that 
performance  at  retention  was  more  dependent  on  individual  performance  differences  at  completion  of 
initial  training  than  on  the  length  of  the  retention  interval,  training  workload,  or  amount  of  training. 
Goldberg  and  O’Rourke  (1989)  did  not  determine  a  function  to  predict  retention  performance  based  on 
training  performance  but  suggest  that  this  would  be  a  viable  approach. 

Arthur  et  al.  (1998)  also  advocated  a  correlational  approach  and  attempted  to  determine  quantitative 
estimates  of  the  effect  sizes  of  a  number  of  potential  predictors  of  skill  retention,  including: 
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•  Length  of  retention  interval. 

•  Degree  of  overlearning. 

•  Task  characteristics: 

>  Closed-  versus  open-loop. 

>  Physical  versus  cognitive. 

>  Natural  versus  artificial. 

>  Speed  versus  accuracy  demands. 

•  Methods  of  testing. 

•  Conditions  of  retrieval. 

•  Evaluation  criteria 


1.  Conditions  of  retrieval 

2.  Speed  versus  accuracy  task 

3.  Physical  versus  cognitive  task 

4.  Closed-  versus  open-loop  task 

5.  Evaluation  criteria 

6.  Retention  interval 

7.  Method  of  testing 

8.  Degree  of  overlearning 

9.  Natural  versus  artificial  task 


Table  4.8.  -  Arthur  et  al.’s  (1998)  Ranking  of  Factors  Affecting  Skill  Retention 


They  performed  a  meta-analysis  of  empirical  findings  to  generate  aggregate  effect  sizes  across  52 
relevant  articles  containing  189  independent  data  points.  This  provided  an  indication  of  the  overall 
magnitude  of  skill  loss  over  retention  intervals.  Each  of  the  factors  had  significant  effects  on  retention 
performance,  as  reviewed  earlier  in  Section  4.2.  To  use  these  factors  as  predictors,  Arthur  et  al.  (1998) 
calculated  effect  sizes  using  the  d  statistic.  This  is  a  measure  of  the  strength  of  treatment  of  the 
independent  variable  and  is  the  standardized  difference  between  two  means.  It  represents  the  observed 
difference  between  experimental  and  control  group  performance  in  standardized  deviation  units 
(Cohen,  1990,  cited  in  Arthur  et  al.,  1998).  The  rank  order  of  factors  by  effect  size  is  presented  in 
Table  4.8.  This  ranking  differs  from  others  that  have  been  published  (Farr,  1987,  cited  in  Arthur  et  al., 
1998)  but  the  authors  argue  that  their  ranking  is  the  only  one  based  on  a  quantitative  assessment  of 
effect  size. 

The  next  step  in  Arthur  et  al.’s  (1998)  analysis  was  to  convert  estimates  of  effect  size  ( d  statistic)  into 
predictions  of  retention  in  units  of  time.  For  example,  the  amount  of  skill  loss  associated  with  the 
retention  interval  ranged  from  a  d  of  -0. 1  immediately  after  training  to  a  d  of  -1.4  after  more  than  365 
days.  Based  on  this  value,  Arthur  et  al.  (1998)  calculated  that,  after  365  days,  an  average  individual 
would  perform  at  roughly  8%  of  his  or  her  initial  performance  level  after  training.  Similar 
calculations  can  be  made  for  any  factor  based  on  the  d  statistics  determined  for  them. 

Arthur  et  al.  (1998)  had  intended  to  generate  skill  retention  curves  based  on  each  factor  to  take  into 
account  the  effect  of  retention  interval  for  each  factor.  They  were,  however,  unable  to  compute  the 
necessary  parametric  statistics  because  the  assumption  of  normality  in  the  data  was  severely  violated. 
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Thus,  this  technique  was  not  able  to  provide  a  usable  set  of  predictions  based  on  the  full  range  of 
relevant  factors. 

A  problem  closely  related  to  prediction  of  skill  retention  is  the  prediction  of  skill  transfer  from  one  job 
to  another.  Here,  the  goal  is  to  estimate  performance  at  a  task  based  on  performance  of  a  related  task. 
Skill  transfer  is  relevant  because  retention  can  be  considered  a  special  case  of  transfer  in  which  a  skill  is 
transferred  from  the  training  environment  to  a  similar  but  distinct  work  environment. 

Kavanagh,  Lance,  O’Brien,  Stennett,  McMillen,  and  Solomonson  (1997)  discuss  development  of  an 
Ease-of-Movement  Matrix  (EOMM),  a  tool  for  estimating  transferability  of  skills  among 
approximately  160  enlisted  Air  Force  Specialties  (AFSs).  Their  approach  could  be  adapted  to 
prediction  of  skill  retention  by  including  in  the  matrix  the  transfer  from  training  to  job  performance. 
Transfer  of  skill  is  estimated  in  terms  of  the  amount  of  retraining  needed  when  an  airman  transfers 
from  one  AFS  to  a  new  one.  The  estimated  retraining  times  within  the  EOMM  would  be  organized  by 
rows  indicating  180  “From”  AFSs  and  columns  indicating  180  “To”  AFSs. 

Kavanagh  et  al.  (1997)  developed  a  Transfer  of  Skills  Questionnaire  (TSQ)  for  retrainees, 
supervisors/peers,  and  nontrainees.  The  TSQ  assessed  the  number  of  months  a  trainee  served  in  his  or 
her  present  assignment,  measures  of  actual  time-to-proficiency  of  the  trainee,  measures  of  current  job 
performance,  and  a  measurement  of  transfer  of  skills  developed  by  Lance  et  al.  (1993,  cited  in 
Kavanagh  et  al.,  1997).  The  questionnaire  was  given  to  a  sample  of  3000  retrainees,  their  supervisors 
and  peers,  and  a  set  of  nontrainees.  The  results  indicated  a  significant  convergent  validity  for  each 
measure  representing  the  underlying  retraining-related  variables.  Correlations  between  cross-job 
retraining  times  indicated  that  these  times  are  valid  predictors  of  actual  ease  of  retraining  but  not 
retraining  success  following  reassignment.  Retrainee  aptitude  seemed  to  play  a  larger  role  in 
determining  job  performance  than  what  was  learned  during  retraining.  Thus,  questions  remain  about 
the  validity  of  the  EOMM  as  a  predictor  of  true  skill  transfer. 

Perhaps  the  most  radical  of  the  correlational  approaches  is  one  that  involves  ignoring  the  causal  factors 
of  skill  loss  altogether.  Sulzen  (1997)  conducted  a  study  to  determine  the  effects  of  home  station 
training  on  performance  at  the  US  National  Training  Center  (NTC).  The  results  indicated  that  units 
expending  more  resources  during  home  station  training  performed  better  at  the  NTC.  Performance  at 
the  NTC  was  essentially  a  retention  test,  albeit  one  at  a  relatively  short  retention  interval.  The  results 
suggested,  however,  that  an  actuarial  approach  might  offer  accurate  prediction  of  unit  performance 
levels  over  time.  In  this  case,  one  would  seek  to  estimate  performance  based  on  unit  data,  including 
resources  used  for  training.  Training  expenditure  is  an  indirect  indication  of  initial  learning  but  is 
easily  and  unambiguously  measurable. 

4.6.2  Applicability  and  Practicality 

Empirically  derived  functions  can  offer  highly  accurate  prediction  of  skill  retention  for  any  kind  of 
skill.  The  major  drawback  of  this  approach  is  that  it  requires  extensive  effort  and  commitment  of 
resources  to  select  a  representative  sample,  administer  training,  and  test  retention  performance  over  a 
suitable  range  of  retention  intervals.  Moreover,  this  process  must  be  performed  for  every  class  of  skill 
for  which  prediction  will  be  made.  Power  functions  will  not  generalize  widely  across  different  classes 
of  skills  and  it  is  necessary  to  carefully  examine  the  applicability  of  a  function  from  one  skill  to 
another.  Given  the  large  number  of  skills  trained  by  the  CF,  it  would  be  a  major  undertaking  to 
develop  empirical  retention  functions  for  all  skills. 

Quantitative  approaches  that  relate  predictive  factors  to  retention  performance  also  tend  to  apply  to  the 
full  range  of  military  tasks,  although  which  factors  most  predictive  presumably  varies  from  task  to 
task.  It  may  be  necessary  to  develop  separate  models  for  different  kinds  of  tasks,  depending  on  the 
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interaction  of  predictive  factors  with  the  type  of  task.  In  particular,  continuous  motor,  procedural,  and 
cognitive  tasks  may  be  optimally  predicted  by  different  sets  of  factors,  or  by  different  weightings  of  a 
single  set  of  factors.  Ultimately,  the  applicability  of  a  quantitative  model  depends  on  the  factors  used 
to  model  skill  retention. 

The  cost  and  workload  associated  with  quantitative  approaches  are  probably  low  because  many  factors 
are  easy  to  assess.  Some  factors,  however,  might  require  that  frequent  measurements  be  made  for  all 
individuals  taking  refresher  training.  One  way  to  deal  with  this  potential  problem  is  to  randomly 
sample  performance  from  a  unit.  This  is  a  basic  technique  of  empirical  research  and,  assuming  proper 
procedures  are  followed,  yields  accurate  and  reliable  estimates  of  levels  of  performance  of  entire  unit. 
Sampling  reduces  the  costs  of  assessing  individual  factors  because  only  a  small  fraction  of  soldiers  are 
directly  assessed.  This  technique  can  dramatically  reduce  costs  without  sacrificing  much  accuracy  as 
long  as  a  sufficient  sample  can  be  drawn  from  the  population.  There  are  two  potential  sources  of 
error:  a)  error  of  measurement,  b)  error  of  sampling.  Reducing  errors  of  measurement  is  a  matter  of 
refining  measurement  techniques.  Errors  of  sampling  can  be  minimized  by  employing  rigorous 
sampling  procedures  to  eliminate  bias  in  the  sample  and  employing  the  technique  for  populations  in 
which  retention  is  normally  distributed  or  by  adjusting  sampling  techniques  for  a  known  distribution 
type. 
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5.  Improving  Skill  Retention  and  Reacquisition 


The  rationale  for  predicting  skill  loss  is  to  enhance  our  capacity  to  reduce  it.  With  increasingly 
accurate  estimates  of  the  degree  to  which  skilled  performance  will  degrade  over  time,  we  can  take 
various  actions  -  better  initial  training,  more  frequent  refresher  training,  provision  of  job  aids,  and  so 
on  -  that  will  preemptively  reduce  or  eliminate  the  decline  in  performance.  This  section  briefly 
discusses  some  trends  that  will  contribute  to  this  effort.  Moreover,  changes  in  training  and  the  work 
environment  may  dramatically  alter  the  course  of  skill  loss.  It  may  be  necessary  to  periodically  review 
and  revise  methods  for  predicting  skill  loss  to  take  into  account  the  changing  nature  of  training  and 
other  factors. 

5.1  Effects  of  Training  Technologies 

The  technology  available  to  trainers  and  training  developers  will  undergo  dramatic  changes  in  the  next 
decade  (Richard,  1997;  Taylor,  1997).  These  changes  hold  the  potential  to  greatly  enhance  the  success 
of  training  and  the  long-term  retention  of  skills.  New  technologies  can  enhance  training  and  practice 
of  military  skills  in  three  basic  ways: 

•  Making  better  use  of  training  methods  and  feedback. 

•  Expanding  practical  methods  of  training. 

•  Permitting  more  frequent  practice. 

Simulations  and  exercises  have  been  a  staple  of  military  training,  especially  for  combat  and  high-skill 
jobs.  Improvements  in  computer  technology  allow  trainers  to  make  exercises  more  realistic  and 
informative  than  in  the  past.  Semi- Autonomous  Forces  (SAF),  for  example,  are  computer-based 
simulations  used  in  tactical  combat  training  (Komell,  1999).  These  allow  personnel  to  train  in  real¬ 
time  settings  against  credible  forces.  SAFs,  unlike  live  exercises,  require  little  time  or  equipment  and 
reduce  the  need  for  training  facilities.  Thus,  personnel  with  access  to  personal  computers  can  engage 
in  training  more  frequently.  Moreover,  simulations  allow  for  training  opportunities  that  are  impossible 
to  create  under  field  training  (Brown,  1992,  cited  in  Sterling  &  Quinkert,  1998). 

Computers  also  expand  the  amount  and  specificity  of  feedback  that  can  be  given  to  trainees.  Brown, 
Nordyke,  Gerlock,  Begley,  and  Meliza  (1998)  note  two  complementary  forms  of  feedback  typically 
used  in  military  training:  a)  intrinsic  feedback  provided  within  exercises  from  the  actual  and  simulated 
entities  and  activities,  and  b)  extrinsic  feedback  provided  by  controllers/trainers  in  the  form  of 
criticism,  reviews,  coaching,  and  correction.  Both  forms  of  feedback  are  enhanced  when  computers 
can  take  over  the  workload  for  providing  feedback.  The  more  realistic  and  complete  the  exercise,  the 
better  is  intrinsic  feedback  because  the  exercise  more  accurately  models  the  cues  and  consequences 
associated  with  performance  of  skills  in  the  work  environment.  Computer  recording  can  enhance  a 
trainer's  ability  to  observe  and  analyse  trainees'  performance  to  provide  more,  and  more  accurate, 
extrinsic  feedback. 

Training  devices,  including  computer  simulations  and  multimedia  training  packages,  reduce  costs 
without  reducing  training  effectiveness  (Komell,  1999).  Indeed,  they  can  improve  training 
effectiveness  while  permitting  trainees  to  engage  in  more  instruction  and  exercises.  As  computers 
become  easier  to  deploy,  these  benefits  can  be  extended  to  units  in  the  field  for  use  in  extended 
training  or  refresher  training.  Taylor  (1997)  notes  that  training  devices  can  be  viewed  in  two  ways,  as 
cheap  training  sessions  and  as  devices  used  by  units  to  slow  the  rate  of  skill  loss,  so  that  training 
sessions  need  be  scheduled  less  often.  In  other  words,  training  devices  can  enhance  training  so  that 
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military  personnel  achieve  higher  levels  of  initial  proficiency  and  enhance  practice  opportunities  so 
that  skills  are  lost  more  slowly.  There  are  many  examples  of  training  devices  already  in  use  by  the  US 
Army,  including  among  others  the  Conduct-of-Fire  Trainer  (COFT)  for  armor,  Precision  Gunnery 
Training  System  (PGTS)  and  Basic  Skills  Trainer  (BST)  for  infantry,  and  the  Precision  Range 
Integrated  Maneuver  Exercise  (PRIME)  for  combined  arms  (see  Taylor,  1997). 

Deploying  computer-based  simulations  and  training  packages  to  the  field  will  allow  units  to  address 
training  deficiencies  and  skill  loss  for  themselves  (Sterling  &  Quinkert,  1998).  The  Graphical 
Instruction  in  LISP  (GIL)  is  one  example  of  a  training  device  that  can  be  used  for  self-teaching 
(Reiser,  1997).  GIL  emulates  a  human  tutor  by  not  only  presenting  lessons  and  exercises  but 
monitoring  and  analysing  trainees'  performance  to  detect  errors  and  bad  strategies.  Thus,  GIL 
provides  individualized  feedback  and  correction  and  can  even  adjust  lessons  to  the  level  of  proficiency 
of  each  trainee.  With  access  to  such  a  training  device,  military  personnel  can  seek  refresher  training 
frequently.  As  more  training  devices  become  available,  units  will  be  able  to  take  advantage  of  the 
features  of  computer  systems,  such  as  their  capacity  to  present  vast  numbers  of  different 
situations/problems,  automatic  scoring  and  feedback,  and  approximation  of  real-world  conditions 
(Taylor,  1997).  Flexible  scheduling  encourages  military  personnel  to  seek  more  practice  opportunities. 

5.2  Military  Work  Environment 

Although  changes  in  training  technology  will  probably  contribute  to  enhanced  learning  and  retention, 
changes  in  the  military  work  environment  -  roles,  missions,  equipment,  etc.  -  may  contribute  to  more 
rapid  skill  loss.  Some  analysts  anticipate  that  the  changing  global  political  and  economic  situation  is 
forcing  the  military  to  take  on  non-traditional  roles  and  methods  of  operating  (O'Neill,  1996).  One 
especially  important  change  has  been  the  increasing  deployment  of  combat  forces  to  Military 
Operations  Other  Than  War  (MOOTW).  Such  deployments  are  of  concern  because  combat  forces 
must  maintain  proficiency  to  wage  war  under  conditions  that  limit  sustainment  of  warfighting  skills 
(Taylor,  1997).  Most  MOOTW  involve  sensitive  political  issues  and  highly-constrained  operations  so 
that  units  rarely  have  access  to  training  facilities.  As  a  result,  developing  new  training  methods  and 
deployable  training  equipment  will  become  extremely  important  to  sustaining  skills. 

Another  problem  will  be  the  increasing  complexity  of  the  operational  environment.  More  and  more,  the 
military  employs  complex  technology.  This  requires  military  personnel  and  leaders  to  learn  more  new 
skills  and  to  perform  and  think  in  a  faster-paced  environment  (Richard,  1997).  The  "21st  century 
soldier"  will  employ  a  great  deal  of  technology  but  will  have  to  think  about  what  the  equipment  is  doing 
and  how  to  use  it  (Salter  &  Black,  1998).  The  most  significant  advances  in  military  systems  are 
combinations  of  improved  hardware,  software,  and  supporting  equipment  collectively  known  as 
digitization  (Ford,  Campbell,  &  Cobb,  1998;  Salter  &  Black,  1998).  The  challenge  will  be  to  make  use 
of  the  significant  benefits  of  digitization  in  a  way  that  assists  the  soldier  rather  than  hindering  through 
increased  workload  and  interference  with  learning  and  performance. 

The  increasing  digitization  within  the  military  increases  the  need  for  well-trained  military  personnel 
and,  hence,  for  training  programs  (Salter  &  Black,  1998).  Digitization  also  places  a  premium  on 
cognitive  skills,  many  of  which  are  complex  and  difficult  to  learn.  Computers  and  complicated 
equipment  are  often  unforgiving  of  errors  so  that  training  must  be  intense  to  achieve  adequate 
learning.  Many  cognitive  skills  are  then  prone  to  rapid  forgetting,  which  increases  the  need  for 
refresher  training  (Elliot  et  al.,  1996,  cited  in  Salter  &  Black,  1998).  As  a  result,  military  personnel 
will  likely  be  required  to  train  more  frequently  and  more  intensely  in  the  future.  Salter  and  Black 
(1998)  note  that  in  Operation  Desert  Hammer  VI,  soldiers  who  had  not  received  adequate  hands-on 
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experience  with  digital  equipment  simply  were  not  proficient  in  using  it.  They  concluded  that  training 
with  actual  equipment  was  needed  for  soldiers  to  learn  and  retain  cognitive  skills. 

Ford  et  al.  (1998)  argue  that  digital  development  outpaces  development  of  training  programs. 
Moreover,  many  units  of  the  US  Army  were  judged  to  be  unprepared  for  the  added  training  load  of 
digitization.  Ford  et  al.  (1998)  note  the  need  to  develop  training  requirements  and  to  allocate 
resources  to  meet  those  requirements. 

Training  in  new  cognitive  skills,  however,  may  hurt  retention  of  other,  more  traditional  skills. 

Learning  new  skills  always  has  the  potential  to  interfere  with  memory  for  previously  learned  skills 
(Adams,  1987).  If  digital  skills  are  given  intense  training,  this  interference  effect  will  naturally  be 
more  severe.  Furthermore,  there  is  only  limited  time  during  which  military  personnel  can  train. 
Scheduling  training  for  new  skills  will  take  away  time  and  resources  that  could  be  spent  training  or 
refreshing  basic  combat  skills. 

5.3  Strategies  for  Sustaining  Readiness 

As  researchers  have  examined  ways  to  predict  skill  retention,  their  ultimate  goal  has  been  to  improve 
the  sustainment  of  performance  or  readiness.  From  the  research  have  come  three  broad  strategies  to 
sustain  readiness: 

1.  Optimize  initial  learning. 

2.  Optimize  prediction  of  skill  retention  for  more  effective  scheduling  of  refresher  training. 

3.  Reduce  the  rate  of  skill  decay. 

The  first  strategy  follows  from  research  on  overlearning,  which  has  demonstrated  that  the  better 
soldiers  perform  immediately  after  training  the  better  they  will  perform  after  a  retention  interval 
(Hagman  &  Rose,  1983;  Hurlock  &  Montague,  1982;  Wisher  et  al.,  1999).  Thus,  without  knowing  or 
attempting  to  adjust  forgetting  rates  or  refresher  training,  one  can  be  confident  that  soldiers  will  stay  at 
a  criterion  level  of  performance  longer  if  they  have  been  given  some  degree  of  overtraining  (Arthur  et 
al.,  1998;  Lance  et  al.,  1998;  Schendel  &  Hagman,  1980,  cited  in  Hagman  &  Rose,  1983).  This  is 
simply  a  consequence  of  the  nature  of  the  forgetting  curve,  which  declines  in  a  predictable  fashion 
from  whatever  initial  performance  level  soldiers  have  achieved.  The  value  of  overlearning  derives 
from  the  finding  that  even  individuals  who  perform  at  a  level  of  100%  accuracy  or  the  minimum 
performance  time  for  a  task  still  benefit  from  further  training,  even  though  this  is  not  evident  in  their 
training  performance  (Driskell  et  al.,  1992).  Thus,  an  individual  who  is  trained  beyond  the  level  of 
perfect  performance  will  exhibit  slower  forgetting  and  better  levels  of  performance  over  time  than  an 
individual  trained  just  to  the  level  of  perfect  performance. 

Overtraining  can  be  performed  all  at  once  during  the  initial  training  session  or  spread  out  over  time  in 
the  form  of  refresher  training  sessions  to  achieve  the  same  effect.  Schendel  and  Hagman  (1982) 
directly  compared  the  effects  of  overtraining  and  refresher  training  on  retention  of  skill  in 
disassembly/assembly  of  the  M60  machinegun.  The  overtraining  group  was  trained  to  criterion  then 
received  100%  initial  overtraining,  whereas  the  refresher  group  was  trained  to  criterion  then  received 
the  same  amount  of  additional  training  midway  through  the  8-week  retention  interval.  Schendel  and 
Hagman  (1983)  found  that  the  refresher  and  overtraining  groups  performed  equally  well  on  retention 
tests  after  the  8-week  period.  Thus,  the  total  amount  of  training  determined  retention  performance, 
regardless  of  the  schedule  according  to  which  the  training  was  given.  Given  that  overtraining  is  as 
effective  as  refresher  training,  it  is  a  technique  that  can  reduce  costs  associated  with  bringing  together 
equipment,  personnel,  and  training  staff.  Military  personnel  and  commanders,  however,  may  question 
the  value  of  overtraining  because  it  appears  counterintuitive  that  continued  training  after  achieving  a 
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criterion  level  of  performance  provides  additional  benefits  (Schendel  &  Hagman,  1982).  This 
technique  can  seem  wasteful  even  though  it  is  actually  a  more  effective  use  of  limited  resources. 

The  benefits  of  overtraining  can  be  obtained  by  techniques  other  than  just  repetitive  practice.  Various 
methods  of  training  can  enhance  the  speed  and  effectiveness  of  training,  depending  on  the  nature  of 
the  tasks  being  training.  Cognitive-oriented  training  techniques,  for  example,  can  lead  to  superior 
memory  performance  for  task  knowledge,  whereas  experiential  or  "leam-by-doing"  techniques  can 
improve  learning  of  procedural  skills  (Healy,  1997). 

It  is  also  possible  to  improve  overall  unit  training  performance  by  applying  selection  techniques  to 
identify  individuals  exhibiting  good  aptitude  for  the  kinds  of  skills  to  be  trained.  Selection  has 
traditionally  been  a  component  of  military  training  and  job  placement,  based  on  the  premise  that 
selection  of  individuals  with  appropriate  or  superior  skills  reduces  training  costs  and  increases  overall 
effectiveness  (Levine  &  Wildzunas,  1999). 

Given  that  skill  loss  will  occur  even  if  military  personnel  are  overtrained,  researchers  have  devoted  a 
great  deal  of  effort  to  a  second  strategy  of  determining  means  to  optimize  the  schedule  of  refresher 
training.  This  is  needed  so  that  military  personnel  will  receive  training  when  needed  to  prevent  them 
from  falling  below  a  criterion  level  of  performance.  One  way  to  accomplish  this  is  by  a  Readiness 
Reporting  System  (RRS)  in  which  units  report  unit  readiness  in  terms  of  ability  to  perform  required 
functions  and  estimates  of  the  training  needed  (Harvey,  1997).  An  RRS,  however,  is  typically  based 
on  the  subjective  estimates  of  each  unit  commander  and  there  are  no  checks  on  the  accuracy  of  these 
estimates  or  that  commanders  use  the  same  factors  as  a  basis  for  their  judgments. 

The  UDA  (Rose  et  al.,  1985a;  1985b)  is  the  most  comprehensive  attempt  to  develop  an  objective 
method  of  predicting  skill  retention.  The  projected  role  of  the  UDA  is  to  accurately  inform  unit 
commanders  of  their  unit  performance  levels  over  time  so  that  refresher  training  can  be  scheduled  in 
an  effective  manner  (Wisher  et  al.,  1999).  The  value  of  the  UDA  compared  to  other  approaches  is  that 
the  UDA  has  been  scaled  to  actual  retention  intervals.  That  is,  the  UDA  provides  predictions  of  the 
effects  of  retention  interval  in  units  of  real  time  so  that  unit  commanders  can  easily  determine  the 
predicted  level  of  unit  performance  at  any  interval  up  to  one  year.  Other  approaches  have  not  been 
refined  to  this  point  and,  as  yet,  can  offer  only  relative  estimates  of  skill  retention. 

The  third  strategy  is  to  reduce  the  rate  at  which  skill  decay.  Some  researchers  (e.g.,  Healy,  1997; 
Sterling  &  Quinkert,  1998)  have  recommended  development  of  computer-based  training  programs  and 
simulations  as  a  means  of  slowing  skill  loss.  Such  training  devices,  when  deployed  to  units,  allow 
almost  continuous  opportunities  for  practice.  A  potential  drawback  of  such  training  devices,  aside 
from  the  cost  of  their  widespread  distribution,  is  that  they  may  be  unable  to  provide  the  range  of 
realistic  scenarios  and  training  practices  needed  to  give  military  personnel  effective  practice  (Brown, 
1992,  cited  in  Sterling  &  Quinkert,  1998).  Advances  in  computer  technology  are  making  computer- 
based  training  more  effective  but  the  applicability  of  this  technique  depends  on  the  particular  task 
under  consideration.  Marx's  (1986)  Relapse  Prevention  Model  (Section  3.4.2)  could  be  a  valuable  tool 
to  provide  individuals  with  techniques  to  reduce  forgetting  and  skill  loss.  This  model  teaches  people 
how  to  recognize  situations  and  behaviors  that  contribute  to  forgetting  and  how  to  combat  these 
factors.  The  model,  however,  is  directed  at  retention  of  managerial  skills  and  it  is  unclear  how  it 
applies  to  other  cognitive  or  procedural  tasks. 

The  US  Army  Europe  (USAREUR)  developed  a  strategy  to  sustain  combat  readiness  while  deployed 
in  Bosnia  for  a  peacekeeping  mission  that  illustrates  these  three  strategies  to  some  degree  (see 
Blankmeyer,  1998).  The  first  step  of  the  USAREUR  strategy  was  to  complete  rigorous  combat  and 
peacekeeping  training  prior  to  deployment  to  Bosnia.  The  home  training  seems  to  have  served  as 
overtraining,  which  "inoculated"  units  against  excessive  skill  loss.  The  second  step  was  to  perform 
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refresher  training  for  warfighting  skills  during  deployment.  This  was  often  difficult  due  to  limited 
access  to  training  equipment  and  locations.  Subordinate  commanders  were  given  substantial  freedom 
to  develop  training  programs  to  make  best  use  of  limited  resources  and  tailored  to  the  particular 
refresher  training  needs  of  their  units.  Units  made  extensive  use  of  simulators  and  other  training 
devices  that  could  be  transported  to  the  field,  which  helped  overcome  some  limitations  imposed  by  the 
politically  sensitive  nature  of  the  peacekeeping  mission.  Finally,  upon  return  from  Bosnia,  units 
engaged  in  retraining  in  combat  skills  and  were  able  to  achieve  combat  readiness  in  somewhat  less 
time  than  anticipated. 
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6.  Discussion 


6.1  Lessons  Learned 

1.  Many  factors  affect  skill  retention.  We  have  documented  a  number  of  these  factors  (Section  3.1) 
but,  in  fact,  there  are  more  factors  than  can  reasonably  be  accommodated  in  either  this  literature 
review  or  a  model  for  predicting  skill  retention.  Thus,  a  first  step  in  predicting  skill  retention  must 
always  be  determining  the  subset  of  factors  to  include  in  the  model  to  provide  maximum 
predictive  power. 

This,  however,  is  not  necessarily  a  straightforward  process.  If  we  could  rank  order  factors  by  the 
degree  of  their  effect  on  skill  retention,  we  would  have  a  means  to  select  the  most  influential 
factors.  Only  a  few  studies,  however,  have  so  far  attempted  to  quantify  and  rank  the  effects  of 
factors  on  skill  retention  (Arthur  et  al.,  1998;  Farr,  1987,  cited  in  Arthur  et  al.,  1998).  These 
studies  have  been  limited  by  the  paucity  of  data.  Arthur  et  al.  ( 1998),  for  example,  were  unable  to 
generate  skill  retention  curves  to  test  the  validity  of  the  negatively  accelerated  forgetting  function 
suggested  by  prior  reviews.  Furthermore,  they  were  unable  to  perform  analyses  of  the  moderator 
effects  of  factors  because  effect  sizes  and  retention  intervals  were  not  normally  distributed. 

Perhaps  a  more  fundamental  obstacle  to  ranking  factors  in  terms  of  their  effects  on  skill  retention 
is  the  likelihood  -  indeed  the  virtual  certainty  -  that  factors  interact  in  complex  ways.  Numerous 
studies  have  demonstrated  interaction  effects  between  various  classes  of  task,  training,  and 
individual  factors  (see  e.g.,  Arthur  et  al.,  1998;  Driskell  et  al.,  1992;  Rose  et  al.,  1985b). 
Depending  on  the  nature  of  these  interactions,  retention  of  some  skills  may  be  best  predicted  by 
one  set  of  factors  and  retention  of  other  skills  by  a  different  set  of  factors.  It  may  be  necessary  to 
develop  a  functional  classification  of  military  tasks  and  study  the  relative  effects  of  factors  for 
each  category.  It  may  be,  for  example,  that  basic  procedural  skills,  specialized  skills,  and 
command  skills  will  be  affected  differently  by  factors  and,  hence,  differentially  predicted  by 
models  of  skill  retention.  It  is,  at  this  point,  an  open  question  whether  all  kinds  of  skills  can  be 
accurately  predicted  by  a  single  model.  The  most  comprehensive  model  available,  the  UDA 
model  (Rose  et  al.,  1985a,  1985b),  was  developed  and  validated  for  a  fairly  restricted  range  of 
specialized  MOS  skills. 

2.  Skill  loss  follows  a  predictable  course.  Despite  the  effects  of  a  large  number  of  factors,  skilled 
performance  declines  according  to  a  power  function  of  time  (Anderson,  1995;  Childs  &  Spears, 
1986;  Wixted  &  Ebbesen,  1991).  Thus,  any  given  factor  influences  the  rate  at  which  performance 
declines  and  the  asymptote  level  at  which  performance  settles  but  does  not  dramatically  alter  the 
basic  nature  of  forgetting.  In  all  cases,  most  skill  loss  occurs  soon  after  training  and  the  rate  of 
forgetting  continually  decreases. 

It  is  important  to  define  skills  clearly  and  accurately  in  order  to  predict  retention.  Most  tasks 
involve  sequences  or  sets  of  distinct  learned  skills.  If  these  are  not  distinguished,  retention  may 
appear  to  follow  a  complex  function  that  deviates  from  the  traditional  power  function. 
Amalgamating  skills  will  necessarily  increase  the  error  in  prediction  of  retention. 

3.  Models  of  skill  retention  tend  to  consider  only  factors  associated  with  the  task.  We  identified  four 
classes  of  factors  relevant  to  skill  retention  (Section  3.1): 
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•  Task  factors,  which  pertain  to  the  nature  of  the  task  or  skill  to  be  acquired;  e.g., 
complexity. 

•  Training  factors,  which  pertain  to  the  types  of  training  manipulations  employed  in 
initial  learning;  e.g.,  length,  style. 

•  Learner  factors,  which  pertain  to  the  individual  differences  and  factors  that  affect 
learning  and  retention;  e.g.,  aptitude,  experience. 

•  Retention  interval  factors,  which  pertain  to  the  events  and  manipulations  occurring 
between  training  and  performance;  e.g.,  length,  opportunity  for  practice. 

These  classes  reflect  pragmatic  rather  than  theoretical  distinctions  and  the  choice  of  task  factors  as 
a  basis  for  predicting  skill  retention  likewise  reflects  practical  concerns.  Task  factors  are,  in 
general,  easier  to  assess  than  other  classes  of  factors  and,  as  such,  less  expensive  in  time,  money, 
and  other  resources  to  use.  Task  factors  are  stable  as  long  as  the  task  itself  does  not  change  and 
predictions  will  apply  equally  to  all  individuals  and  training  groups  performing  the  task.  Training, 
learner,  and  retention  interval  factors  are  dependent  on  the  particular  individuals  receiving  and/or 
conducting  training.  Thus,  these  factors  must  be  assessed  for  each  individual  and  each  training 
group  to  be  used  to  predict  skill  retention.  This  dramatically  increases  the  effort  involved  in  using 
these  factors  as  predictors. 

Although  task  factors  are  clearly  very  convenient,  it  is  not  clear  that  they  offer  the  best  predictors 
of  skill  retention.  Factors  such  as  the  degree  of  overlearning  (Hagman  &  Rose,  1983;  Lance  et  al., 
1998;  Schendel  &  Hagman,  1980,  cited  in  Hagman  &  Rose,  1983),  opportunity  to  practice 
(Blankmeyer,  1998;  Ford  et  al.,  1992;  Landry  &  Campbell,  1997),  similarity  of  training  and 
performance  environments  (Healy  et  al.,  1992;  Marmie  &  Healy,  1995),  and  possibly  individual 
aptitude  (Earles  &  Ree,  1992;  Hurlock  &  Montague,  1982)  have  strong  effects  on  skill  retention. 
To  achieve  the  best  possible  prediction,  a  model  should  include  these  factors. 

4.  Subjective  models  of  skill  retention  have  the  potential  to  lead  to  usable  models  for  predicting  skill 
retention  but  require  further  development.  A  long  tradition  in  cognitive  psychology  supports  the 
power  and  utility  of  human  meta-cognitive  abilities.  Moreover,  a  number  of  studies  (Gordon, 
1991;  Schendel  &  Hagman,  1982;  Wisher  et  al.,  1991)  indicate  that  people's  self-assessments  can 
be  highly  predictive  of  retention  and  need  for  refresher  training. 

The  research  literature,  however,  is  not  uniformly  supportive.  Some  studies  suggest  that  people's 
self-assessments  tend  to  be  biased  (e.g.,  Schendel  &  Hagman,  1982).  Other  studies  have  found 
self-assessments  of  skill  retention  less  accurate  than  assessments  by  observers  (Wisher  et  al., 

1991).  Thus,  it  is  not  clear  that  self-assessment  can  serve  as  the  basis  for  prediction  of  military 
skill  retention  at  the  present  time.  Further  study  and  development  is  needed  to  validate  this 
approach  and  devise  reliable  methods  to  obtain  accurate  self-assessments  of  knowledge  and  need 
for  training. 

There  are  a  number  of  advantages  to  subjective  methods.  They  are  relatively  easy  and  inexpensive 
to  obtain,  involving  minimal  training  and  no  special  equipment.  These  techniques  also  afford  very 
flexible  means  to  monitor  and  remediate  skill  loss.  Using  subjective  techniques,  individuals  could 
potentially  customize  a  schedule  of  refresher  training  that  optimizes  retention  of  skills  for  each 
individual  in  a  unit.  These  advantages,  however,  are  limited  by  the  susceptibility  of  subjective 
techniques  to  malingering.  Without  adequate  safeguards,  individuals  can  avoid  needed  training 
simply  by  claiming  to  have  better  retention  than  they  actually  do.  Likewise,  individuals  could 
engage  in  an  excess  of  unnecessary  training  by  claiming  to  need  it.  These  concerns  can  lead  to  a 
general  mistrust  of  subjective  methods  among  unit  leaders  and  trainers,  which  would  likely  reduce 
their  effectiveness  further. 
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5.  Qualitative  approaches  to  predicting  skill  retention  do  not  seem  well  suited  to  predicting  retention 
of  military  skills.  These  tend  to  be  the  least  developed  kinds  of  models,  which  leads  to  vague 
predictions  of  retention.  These  models  also  tend  to  be  more  prescriptive  than  predictive  (see 
Marx,  1986).  That  is,  they  are  designed  more  to  offer  techniques  and  strategies  to  reduce  skill  loss 
than  means  to  predict  the  rate  of  skill  loss.  Although  such  techniques  are  undoubtedly  valuable, 
they  do  not  address  the  central  concern  of  this  literature  review. 

6.  The  UDA  is  the  model  found  to  be  usable  at  the  present  time  to  predict  retention  of  military  skills. 
Not  only  have  ARI  researchers  identified  a  set  of  predictive  factors,  they  have  determined  the 
empirical  relationships  of  each  factor  to  skill  retention  to  quantify  their  effect  sizes  (Rose  et  al., 
1985a).  These  have  been  scaled  so  that  ratings  can  be  used  to  produce  estimates  of  retention  in 
units  of  real  time. 

The  UDA  model  is  easy  to  use  and  requires  few  resources  in  either  time  or  money.  Some  training 
in  the  use  of  the  UDA  is  necessary  but  Radtke  and  Shettel's  (1985)  study  indicates  that  this 
training  does  not  need  to  be  intense  or  prolonged.  Another  advantage  of  the  UDA  is  that  it  applies 
to  a  range  of  military  skills.  The  model  was  developed  on  the  basis  of  data  gathered  from  U.S. 
Army  populations  performing  various  MOS  tasks.  The  model  was  also  validated  with  respect  to  a 
similar  set  of  data  (Rose  et  al.,  1985b).  The  result  is  a  tool  that  can  be  distributed  to  commanders 
and  trainers  and  employed  with  minimal  training.  No  other  model  surveyed  approaches  this  level 
of  development. 

The  major  weakness  of  the  UDA  model  is  that  ARI  researchers  developed  and  validated  the  model 
with  respect  to  a  somewhat  limited  range  of  tasks.  Although  these  tasks  are  representative  of 
military  activities,  they  were  selected  from  U.S.  Army  MOS  tasks  and,  thus,  represent  specialized 
skills.  These  specialized  skills  tend  to  be  highly  procedural  and  composed  of  discrete  steps.  This 
is  potentially  important  because  the  nature  of  the  task  (e.g.,  discrete  versus  continuous,  motor 
versus  cognitive,  etc.)  can  affect  the  rate  of  skill  loss  (Arthur  et  al.,  1998;  Hurlock  &  Montague, 
1982;  Schendel  &  Hagman,  1991).  The  military  trains  individuals  in  a  wide  range  of  tasks, 
including  basic  tasks  such  as  small  arms  proficiency  and  highly  cognitive  tasks  such  as  decision 
making  and  command.  It  is  unclear  how  well  the  UDA  model  applies  to  these  kinds  of  tasks  and 
we  found  no  study  directly  examining  this  issue. 

Another  problem  is  that  the  UDA  model  has  received  only  a  small  amount  of  empirical  support. 
Many  studies  claim  to  report  evidence  of  the  validity  of  the  UDA  model  (Macpherson,  Patterson, 

&  Mirabella.,  1989;  Sabol,  Chapell,  &  Meiers,  1990;  Wisher  et  al.,  1991;  Wisher,  Sabol,  & 
Ozkaptan,  1996;  see  Wisher,  Sabol,  &  Ellis,  1999)  but  these  do  not,  in  fact,  compare  predictions 
of  the  UDA  to  actual  retention. 

Although  well  developed,  the  UDA  model  requires  more  extensive  validation  based  on  independent 
data  sets  covering  a  comprehensive  range  of  military  tasks.  Empirical  studies  in  this  regard  will 
determine  the  applicability  of  the  model  to  basic,  specialized,  and  command  skills. 

7.  Other  quantitative  approaches  to  predicting  skill  retention  are  based  on  the  same  logic  as  the  UDA 
model  but  are  not  as  well  developed.  A  number  of  studies  have  attempted  to  determine  the 
function(s)  relating  one  or  more  factors  to  skill  retention  (see  Arthur  et  al.,  1998).  Any  factor 
could  potentially  be  related  to  retention  in  this  way;  the  trick  is  to  find  the  most  predictive  and 
usable  factors  for  prediction.  Thus,  quantitative  approaches  do  not  differ  in  principle  from  the 
UDA  model. 

The  main  value  of  other  quantitative  models  is  to  suggest  useful  factors  that  may  be  considered  as 
a  means  to  augment  the  UDA  model.  There  is  no  guarantee  that  adding  more  factors  to  the  UDA 
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model  will  yield  dramatically  better  predictions  but,  as  mentioned,  many  factors  affect  skill 
retention.  In  particular,  it  may  be  worth  considering  training  and  individual  factors  in  addition  to 
task  factors,  despite  the  expense  and  difficulty  of  assessing  these  factors.  Some  quantitative 
approaches  also  suggest  factors  not  traditionally  considered  in  modeling  skill  retention,  such  as  the 
amount  of  resources  expended  on  training  (Sulzen,  1997),  as  a  way  to  improve  predictive  power. 

8.  Changes  in  technology  and  the  military  environment  will  complicate  future  training  requirements. 
We  expect  military  personnel  will  train  in  a  much  wider  range  of  skills  in  the  future  due  to 
changes  in  technology  and  the  military  environment.  Computers  and  computerized  equipment  are 
taking  a  more  prominent  place  in  the  military  and  this  increases  the  need  for  digital  skills 
(Richard,  1997;  Salter  &  Black,  1998).  The  military  is  also  being  called  upon  to  perform  more 
non-traditional  missions,  often  broadly  categorized  as  MOOTW.  These  missions  entail 
specialized  policing  and  support  skills  outside  of  combat  skills  (Taylor,  1997).  Overall,  military 
personnel  will  likely  train  in  more,  and  more  diverse,  skills  than  in  the  past.  This  in  turn, 
complicates  the  problem  of  maintaining  readiness  in  all  trained  skills. 

Although  training  will  become  more  challenging  in  the  future,  advances  in  technology  and 
training  methods  should  aid  sustainment.  Better  training  techniques  can  allow  military  personnel 
to  achieve  higher  levels  of  initial  skill  acquisition,  which  translates  into  better  retention  over  time 
(e.g..  Brown  et  al.,  1998).  The  use  of  mobile  training  devices,  including  simulators  and 
computerized  trainers,  can  allow  for  more  frequent  and  flexible  practice  and  refresher  training  in 
the  field.  The  placement  of  such  devices  heightens  the  need  for  an  accurate  tool  to  predict  skill 
loss  in  order  to  effectively  schedule  practice  and  refresher  training. 

6.2  Recommendations 

1.  Validate  the  UDA  for  the  CF.  Given  the  limited  direct  empirical  support  for  the  UDA,  it  is 
necessary  to  conduct  further  studies  with  independent  data  to  determine  how  effective  it  is  as  a 
predictor  of  skill  retention.  Validation  studies  are  needed  to  ensure  that  the  UDA  can  accurately 
predict  retention  for  basic  and  cognitive  tasks  that  differ  from  the  MOS  tasks  used  in  developing 
the  UDA.  These  studies  will  also  determine  whether  the  UDA  predicts  CF  tasks  to  an  acceptable 
level  of  accuracy  so  that  predictions  are  useful  in  scheduling  refresher  training. 

2.  Should  validation  studies  indicate  that  the  UDA  is  not  sufficiently  predictive,  conduct  research  to 
determine  what  factors  can  be  incorporated  to  increase  the  accuracy  of  predictions.  We  have 
identified  four  potential  factors  (degree  of  overlearning,  opportunity  to  practice,  similarity  of 
training  and  performance  environments,  and  individual  aptitude)  but  it  is  unclear  to  what  extent 
these  factors  are  complementary  to  the  task  factors  assessed  by  the  UDA. 

3.  Develop  self-assessment  tools.  Subjective  techniques  offer  potentially  practical  and  accurate 
means  to  predict  skill  retention.  These  techniques,  however,  must  be  carefully  developed  and 
validated  to  develop  usable  tools  to  aid  scheduling  of  refresher  training. 

4.  Determine  the  applicability  of  the  Relapse  Prevention  Model  to  command,  administrative,  and 
other  related  tasks.  Although  the  Relapse  Prevention  Model  is  unlikely  to  be  broadly  applicable,  it 
may  be  useful  in  predicting  and/or  remediating  skill  loss  in  certain  areas.  Many  command  skills  in 
the  CF  could  share  common  features  with  the  managerial  skills  for  which  the  model  was 
developed. 

5.  Conduct  studies  to  derive  empirical  retention  curves  for  critical  tasks.  Deriving  empirical 
retention  curves  requires  a  large  effort  and  so  this  technique  is  not  broadly  applicable. 
Nevertheless,  empirical  retention  curves  may  be  the  most  accurate  tool  for  predicting  skill  loss 
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available.  Consequently,  where  the  importance  of  a  task  warrants  the  effort,  researchers  should 
develop  retention  curves. 
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This  example  was  taken  from  B-GL-304-002/PT-Z04,  Individual  Battle  Task  Standards  (IBTS), 
Training  for  War,  Volume  2. 

Individual  Battle  Task  Standard  -  Fire  the  C7/C8 

1.  Battle  Task.  Fire  the  C7/C8 

2.  Conditions: 

a.  Given: 

(1)  C7/C8  as  per  EIS, 

(2)  ammunition, 

(3)  fighting  order, 

(4)  fire  control  orders, 

(5)  suitable  firing  area. 

b.  Denied:  assistance. 

3.  Standards.  The  soldier  must  achieve  the  standards  in  accordance  with  reference  A,  prior  to 
deploying  on  operation: 

a.  Pass  weapon  handling  tests. 

b.  Successfully  achieve  the  standard  at  Stage  three  as  follows: 

(1)  infantry  -  zero  and  pass  PWT  level  three  and  night  supplement, 

(2)  others  -  zero  and  pass  PWT  level  two  and  night  supplement. 
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c.  Adopt  firing  positions. 

d.  Engage  a  figure  1 1  target  with  ten  rounds  in  TOPP  high  from  200  m  in  the  prone 
position  and  achieve  (5)  hits. 

e.  TsOET.  The  following  is  the  TsOET  to  be  achieved  yearly  for  this  Battle  Task: 

(1)  pass  the  weapon  handling  tests; 

(2)  adopt  firing  positions; 

(3)  group,  zero  and  pass  Stage  Three,  PWT  level  one  in  accordance  with 
reference  A,  less  the  standing  unsupported  practice. 

4.  Supporting  Task.  The  successful  completion  of  the  following  supporting  tasks  necessary  for 
the  attainment  of  the  standard: 

a.  identifying  and  handling  small  arms  ammunition; 

b.  holding,  aiming  and  applying  the  principles  of  marksmanship; 

c.  maintain  the  C7/C8  to  include: 

(1)  basic  stripping  and  assembly, 

(2)  cleaning. 

5.  Resources.  The  following  resources  are  required  to  meet  thistask  standard: 

a.  222  Rds  of  5.56mm  ball  for  Stage  Three  level  three  PWT  with  night  supplement; 

b.  133  Rds  of  5.56mm  ball  for  Stage  Three,  level  two  PWT  with  night  supplement; 
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c.  35  Rds  of  5.56mm  ball  for  Stage  Three,  level  one  PWT; 

d.  10  Rds  of  5.56mm  ball  for  TOPP  High  shoot; 

e.  30  Rds  of  5.56mm  ball  for  zeroing  IAW  practice  4,  Annex  A  of  reference  A; 

f.  40  Rds  of  5.56mm  ball  for  grouping  IAW  Stage  one,  practice  two  or  three  (range 

dependent)  of  reference  A; 

g.  appropriate  cleaning  and  range  materials. 

6.  Award.  Completion  of  the  TsOET  for  this  Battle  Task  and  achieving  one  of  the  following 
shooting  levels  will  qualify  as  successful  completion  of  the  TsOET  for  WARRIOR  programme 
purposes.  (Padres  are  not  reqr  to  fire  the  PWT  to  be  awarded  a  badge,  but  must  complete  the  handling 
tests)  : 

a.  51  of  60  points:  Gold; 

b.  47  of  60  points:  Silver; 

c.  42  of  60  points:  Bronze. 

7.  References.  The  following  are  the  principal  references  to  be  used  for  this  battle  task: 

a.  B-GL-3 18-006/PT-004  -  Shoot  to  Live, 

b.  B-GL-3 17-018/PT-001  -  Weapons,  Volume  18,  Rifle  5.56mm  C7  and  Carbine 
5.56mm  C8. 
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This  example  was  taken  from  Infantry  Battle  Task  Standards,  B-GL-383-002/FP-015  (INTERIM). 


Infantry  Battle  Task  Standard  4001  B/C  -  ATTACK 

ELEMENT:  SECT  /  PL 
CONDITIONS: 

The  section  /  platoon  commander  reacts  to  effective  enemy  fire,  and  conducts  a  quick  attack.  The 
enemy  has  been  in  position  for  24  hours  and  may  have  prepared  positions  with  mines  and  wire.  Both 
friendly  and  enemy  forces  have  indirect  fire  capabilities.  When  a  well-prepared  enemy  defence  must 
be  defeated,  a  deliberate  attack  may  be  required,  with  emphasis  on  planning  and  preparation  at  the 
expense  of  speed  and  time.  In  this  case,  the  enemy  location  may  or  may  not  be  known,  and  the  enemy 
may  or  may  not  have  engaged  the  friendly  force. 

TASK  STANDARD: 

The  section  /  platoon  successfully  conducts  a  quick  or  deliberate  attack.  The  section  /  platoon  captures, 
destroys  or  forces  the  withdrawal  of  the  enemy  LAW  the  assigned  mission. 

TASKS  AND  STANDARDS 


1 .  The  section  takes  action  LAW  Section  Battle  Drill  Two,  'Reaction  to  Effective  Enemy  Fire,' 
and  LAW  Section  Battle  Drill  Three,  Locates  the  Enemy.' 

2.  Elements  not  in  contact  help  to  determine  essential  enemy  information: 

A.  location  of,  and  covered  and  concealed  routes  into  enemy's  flanks  and  rear 

B.  possible  locations  of  obstacles  and  supporting  enemy  elements 

C.  all  information  on  the  enemy  and  terrain  is  reported  to  the  section  /  platoon 
commander  and  relayed  to  the  higher  HQ. 

3.  If  the  section  is  operating  independently,  the  section  commander  decides  if  the  section  has 
sufficient  combat  power  to  destroy  the  enemy,  or  if  the  section  should  dig  in,  bypass,  or 
withdraw. 

4.  If  the  section  is  part  of  a  platoon,  the  platoon  commander  takes  action  IAW  Platoon  Battle 
Drill  One,  'Reaction  to  Section  coming  under  Effective  Enemy  Fire,'  and  moves  forward  to 
assess  the  situation  and  takes  over,  or  orders  the  section  commander  to  attack. 

5.  If  the  section  is  operating  independently  and  has  sufficient  combat  power  to  destroy  the 
enemy,  or  on  order  from  the  platoon  commander  to  attack,  the  section  commander  completes  a 
combat  estimate,  issues  battle  orders,  and  takes  immediate  action  LAW  Section  Battle  Drills: 

A.  Battle  Drill  Four  -  Winning  the  Fire  Fight' 

B.  Battle  Drill  Five  -  'Approach' 

C.  Battle  Drill  Six  -  'The  Assault' 

D.  Battle  Drill  Seven  -  'Consolidation.' 


Humansy stems  Incorporated 


Skill  Fading  Literature  Review:  Annex  B 


Page  B-3 


-J  ON 


Annex  B: 

Infantry  Battle  Task  Example 


Additional  fire  support  for  the  section  attack  may  be  obtained  from  the  platoon.  OR 
If  the  platoon  commander  decides  to  conduct  a  platoon  attack,  he  completes  a  combat 
estimate,  issues  battle  orders,  and  takes  action  IAW  Platoon  Battle  Drills: 

A.  Battle  Drill  Two  -  'The  Hasty  Attack' 

B.  Battle  Drill  Three  -  'Consolidation.' 

8.  The  section  /  platoon  exploits  any  opportunity  that  may  exist,  but  does  not  proceed  beyond  the 
limit  of  exploitation. 

9.  The  section  /  platoon  commander  reports  the  situation  to  higher  HQ  and  continues  the 
mission. 
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This  example  was  taken  from  Battle  Group  Battle  Task  Standards,  B-GL-383-002/PT-003. 


BATTLE  GROUP  BTS  2001  F  -  EMPLOY  INDIRECT  FIRE  SUPPORT 

ELEMENT:  BATTLE  GROUP 
CONDITIONS 

The  BG  is  operating  as  part  of  the  bde  gp.  Both  en  and  friendly  forces  have  indirect  fire  and  CAS 
aval. 

TASK  STANDARDS 

The  BG  comd  and  BC,  and  cbt  tm  comds  and  FOOs  plan  and  execute  indirect  fire  that  supports  the 
comd's  plan  and  covers  likely  en  approaches  and  key  terrain.  Friendly  fire  does  not  cause  unnecessary 
friendly  casualties. 

BATTLEGROUP  TASKS 

1.  The  BG  comd,  BC  and  FC  (if  aval),  and  cbt  tm  comds  and  FOOs  make  a  tentative  fire  plan: 

a.  develop  a  fire  sp  plan  in  concert  with  the  bde  gp  comd's  plans,  guidance  and  intent 

b.  integrate  the  BC,  FOOs  and  FCs  (if  available)  to  ensure  they  are  in  position  to  execute 

their  assigned  tasks 

c.  augment  the  fire  of  the  bde  gp's  fire  sp  plan  LAW  the  bde  gp  comd's  guidance  to 
ensure  indirect  fire  is  planned  in  depth 

d.  establish  recorded  targets  on  likely  en  approaches  and  positions. 

2.  The  BG  comd  and  BC,  and  cbt  tm  comds  and  FOOs  conduct  a  recce  and  plan  targets: 

a.  identify  en  positions  and  likely  positions 

b.  identify  vital  ground  and  key  terrain 

c.  target  en  approaches 

d.  establish  DFs  and  FPFs 

e.  approve  fire  plan  and  distribute  down  to  pl/tp  comds. 

3.  Execute  fire  sp: 

a.  all  available  supporting  direct  and  indirect  fire  is  synchronized  and  executed  in  a 
timely  manner  and  accomplishes  the  desired  result  IAW  the  fire  plan 

b.  fire  hits  critical/vulnerable  area  of  en  formations  or  positions 

c.  friendly  fire  stops  or  slows  en  movement 

d.  friendly  fire  sufficiently  suppresses  en  fire 

e.  the  volume  of  fire  is  appropriate  for  the  task 

f.  friendly  fire  causes  no  unnecessary  friendly  casualties 

g.  appropriate  arty  control  measures  are  used  to  limit  friendly  casualties 

h.  friendly  forces  are  not  silhouetted  by  friendly  smoke 

i.  fire  sp  priorities  are  adjusted  as  the  battle  progresses 

j.  the  BG  comd  and  BC  effectively  employ  CAS,  attack  helicopters  and  naval  gunfire, 
when  allocated,  and  consider  the  en  air  defence  threat 

k.  engagement  results  are  reported  at  the  conclusion  of  each  fire  msn 

l.  friendly  force  locations  are  reported  to  the  DS  battery  CP. 
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